跳到正文
原文
dair_ai· @dair_ai · X·· 4 小时前AI 评分56

论文提出将决策模型集成进 Agent harness,蒸馏 4B 模型超越 Jev 且成本更低

AI 导读

论文提出 JevAgentHarness,将决策模型集成进 Agent harness:把每个常规选择交给决策模型,仅当置信度超过基于 Agent 自身轨迹拟合的阈值时才采用其答案,否则由 LLM 正常决策。

正文 · 原文

Must-read paper integrating decision models like Jev into harnesses.

They distilled an open 4B decision model, mostly from Jev's answers on the agent's own states, that beats Jev inside that agent and is much cheaper to run.

Their harness, JevAgentHarness, sends each routine choice to the decision model and uses its answer only when the confidence clears a threshold fitted on the agent's own trajectories. Below the threshold, the LLM decides as usual. With the fitted threshold, tool and argument cascades cut cost by 14.2% without changing the outcome of any of 500 tasks. A fixed threshold of 0.5 lost 21.5 points.

They also train AgentDecision-4B on states the agent reaches after its own mistakes, with Jev's answers as labels. On tasks unlike its training data, the agent solved 91.5% with it and 82.0% with Jev. On familiar tasks, 4B models trained this way made agents 81.1 to 89.0% cheaper and up to 2.5 times faster while staying within 3 points of success.

Paper: academy.dair.ai/papers/jevag…

来源:dair_ai · x.com