DAIR.AI 呼吁构建 evals
DAIR.AI 正在开放式世界领域开展模型改进与评测研究，并呼吁开发者学习构建 evals。作者指出，AI 当前擅长的是可验证问题，开放式任务需要不同于短视 benchmark 刷分的方法。他正与该领域建设者交流，欢迎就合作机会或观点展开联系。
Learn to build evals, folks!
You are not RSI-pilled enough if you don't know how to properly measure the thing you want to improve.
It doesn't matter if it's a model, complex codebase, or a business function. This is extremely hard in the real world.
You ever wonder why most of what AI is great at today is around verifiable problems?
The open-ended world requires a completely different approach to improving and measuring models.
We need to go back to hardcore science and evals. Enough of this short-term benchmaxxing.
This space offers so much opportunity. We are doing research and building evals in this space @dair_ai. I am currently chatting with folks building in this space too. Feel free to reach out for potential collab opportunities or just share perspectives.
来源：omarsar0 · x.com