Anthropic 介绍 Claude 对话自评安全评估技术
Anthropic 推出自评技术，让 Claude 在每次回复后对对话进行初步安全评估。在 Claude Fable 5 上，自评以 1% 误报率捕获了提示词分类器标记为潜在有害对话的 78%，以及高优先级对话的 94%。Anthropic 还指出，每次成功攻击消耗的生成 token 约为无安全防护时的 4 倍。
Anthropic introduced self-assessment, a new technique that lets Claude give each conversation a preliminary safety assessment right after it responds.
> On Claude Fable 5, self-assessment flags 78% of the conversations that a stronger, prompted classifier flags as potentially harmful, and 94% of the high-priority ones, at a 1% false-positive rate.
Anthropic also shed light on how self-assessment and prompted classifiers help block attacks.
> "Each successful attack costs roughly four times as many generated tokens as with no safeguards."
We already live in a world where your security depends on the compute allocated to defend the systems you rely on.
来源：testingcatalog · x.com