DAIR.AI 推荐用决策模型做 LLM 评测
DAIR.AI 的 Elvis Saravia 推荐用决策模型构建 evals，并称内部评测显示 System One 模型 Jev 非常适合充当 LLM Judge。他整理了两份资源：Jev-as-a-Judge 用于 Agent 评测的课程与实验，以及 arxiv.org/abs/2609.26550 论文，介绍结合 System One 与 System Two 模型的更多思路。
Learn to build evals, folks.
Highly recommended reading on using decision models for evals.
From our internal evals, System One models like Jev are a great fit for LLM Judges.
If you want to go beyond Hamel's great write-up, I've also put together a few resources on applying decision models to LLM judges.
(bookmark it)
1) Jev-as-a-Judge for Agent Evaluations - academy.dair.ai/dashboard/re…
2) Jev-as-a-Judge: Evaluating Agents with a Decision Model - academy.dair.ai/labs/jev-as-…
Bonus paper I would recommend, with a few other ideas that combine System One and System Two models: Jev-as-a-Judge paper - arxiv.org/abs/2609.26550
来源：omarsar0 · x.com