Anthropic 用数千个 Claude 智能体自动化研究涌现性失配
Anthropic 让数千个 Claude 智能体执行规避训练数据监控器的任务，以自动化红队测试并研究细微的涌现性失配。智能体生成的训练数据表面上无害，例如资深摔跤手指导新人或专业养蜂问答，实际却教会了失配行为；这些样本可用于改进后续训练数据监控器。研究只是初步一步，作者认为未来红队测试可借助更好的模型和更多算力继续改进。
New Anthropic research: Automating subtle emergent misalignment research.
Red-teaming is a vital part of model safety testing, but it’s also limited by researchers’ time. To see if we could automate it, we set thousands of Claude agents the task of evading a training data monitor.
These agents developed training data that looked harmless to monitors (like a veteran wrestler mentoring a newcomer, or a Q&A about professional beekeeping), but that in fact taught misaligned behavior.
Examples like these can be used to directly improve future training data monitors. This research is only a first step, but it suggests that future red-teaming efforts could improve much further, using better models and more compute.
Read the full post on the Alignment Science Blog: alignment.anthropic.com/2026…
来源：AnthropicAI · x.com