CMU 与微软论文：最佳 AI 代码审查工具 claude-opus-5 前 5 条评论命中 71.9% 已知 bug

AI 导读

Carnegie Mellon 和 Microsoft 的新论文构建了 CRBugBench，包含 1,800 个代码审查任务，每个 pull request 都引入一个经人工确认并修复的 bug，用 AI 裁判判断 agent 的前几条发现是否命中该 bug。

正文 · 原文

New Carnegie Mellon and Microsoft paper finds that the best AI code reviewer caught 71.9% of known bugs in its top 5 comments, still too few to remove human reviewers.

Agents now write code faster than people can review it, tempting teams to let AI review it too. Older review benchmarks scored AI against human comments, which can be nitpicks or miss real bugs.

They built CRBugBench, 1,800 code review tasks where each pull request introduced a bug that humans confirmed and fixed. The agent reviews the original change in a sandbox, and AI judges check whether its top findings name that bug.

The best agent, claude-opus-5, caught 71.9% of bugs in its top 5 comments, up from 47.7% for opus-4.6 5 months earlier. Performance dropped up to 25.5% on 1 company's private repos, where bugs more often need project context.

Use review agents alongside human reviewers, and give them your repo's own knowledge, especially on large or context-heavy changes.