研究测试六个前沿模型的研究智能体是否具备研究品味
一项研究让六个前沿模型基于七篇近期 NLP 论文提出并运行实验，以考察研究智能体是否具备研究品味。415 个实验中有 194 个被判定为合理、有价值且新颖，其中 73 个超出种子论文且不显而易见；模型首选方案优于随机选择，但未证明优于最先写出的实验，预测也未明显超过仅阅读论文和实验登记的新实例，多智能体团队则未带来清晰的提案质量提升。论文链接为 arxiv.org/abs/2610.40488。
Regarding autoresearch agents, this question has attracted a lot of attention.
Do research agents have taste?
This has attracted massive interest because autoresearch agents could unlock major discoveries.
This study gave six frontier models seven recent NLP papers, then ran every experiment they proposed, including the ones they ranked last.
Of 415 experiments, 194 were judged sound, valuable, and new. Only 73 were also beyond the seed paper and non-obvious.
The rankings added little. First choices beat a random pick, but they were not shown to beat the experiment written first.
Forecasts beat a base rate, but they were not measurably better than a fresh instance of the same model that read only the paper and registrations.
They tested with teams of agents, which cost more, with no clear improvement in proposal quality.
arxiv.org/abs/2610.40488
Chat with Paper: academy.dair.ai/papers/rtast…
来源：omarsar0 · x.com