Alibaba 与 Cornell 研究编码智能体框架中的验证设计
Alibaba 与 Cornell 比较了 7 个前沿编码模型上的 17 种验证设计，评估其在 Terminal-Bench 和 SWE-bench Verified 上的效果与成本。
Great paper from Alibaba and Cornell on verification in coding-agent harnesses.
(bookmark it)
If you build agent harnesses, this one is worth your time.
They find that a separate reviewer agent adds 5.9 to 9.5 points on Terminal-Bench at 1.13 to 1.25x the baseline cost.
They compared 17 verification designs across seven frontier coding models on Terminal-Bench and SWE-bench Verified. The designs check the agent's work at four points: the initial instructions, reminders during the run, a gate before the agent can finish, and an independent critic at the end.
Instructions and reminders gave small, model-dependent gains. Completion gates gave larger gains, sometimes at much higher cost. The critic gave the best accuracy per dollar, adding 3.9 to 6.6 points on SWE-bench Verified at almost no extra cost. More review rounds added up to 3.5 more points on Terminal-Bench.
arxiv.org/abs/2610.34577
Chat with Paper: academy.dair.ai/papers/rethi…
来源：dair_ai · x.com