FDE-Gym：AI 智能体模拟前沿部署工程师的评测基准
FDE-Gym 是一个将 AI 智能体作为前沿部署工程师（FDE）进行测试的基准，任务均来自真实客户交付项目，要求智能体在类生产系统中质询预设的利益相关方并找出客户未明说的约束。
// FDE-Gym: Agents as Forward-Deployed Engineers //
They found that the best agent configuration fully resolved 14 of 95 real customer-delivery tasks.
FDE-Gym tests agents as forward-deployed engineers. Each task comes from a real delivery project. Agents must question scripted stakeholders, work in production-like systems, and find constraints the customer never stated.
A delivery is scored by how much it improves the customer's own success metric, with penalties for extra messages, trial runs, and waiting time.
Across 13 configurations from seven providers, the top mean score is 0.56. The main failure is missing unstated constraints. Even on 23 tasks where the best configuration collected every key fact, it reached 0.8 or higher on only 10.
Highly recommended benchmark to look at.
arxiv.org/abs/2610.22350
Chat with Paper: academy.dair.ai/papers/fde-g…
来源：dair_ai · x.com