智能体 RL 数据合成 scaling laws 论文解读
DAIR.AI 推荐一篇关于智能体强化学习（agentic RL）数据合成 scaling laws 的论文（arxiv.org/abs/2610.23946），沿任务、环境、轨迹三个轴拟合 scaling law，发现跨模型规模、模型族、领域及 PPO 和 GRPO，指数排序一致：环境 > 任务 > 轨迹。
Brilliant paper on scaling laws for agentic RL data.
It's a great paper worth reading if you are synthesizing data for agentic RL.
(bookmark it)
They fit scaling laws for agentic RL data along three axes: tasks, environments, and trajectories. Across model sizes, families, domains, and both PPO and GRPO, the exponents order the same way, with environments above tasks and tasks above trajectories.
The environment advantage seems to come mainly from world diversity, meaning genuinely different settings. Varying the structure of the same world helps about as much as adding tasks.
The laws also give each synthesis method a discount factor and a synthetic-to-real ratio at which training starts to collapse. Those ratios differ by an order of magnitude across axes.
In addition, allocating the budget with their recipe gives up to 1.7x compute efficiency over the common practice of generating more trajectories.
arxiv.org/abs/2610.23946
Chat with Paper: academy.dair.ai/papers/how-d…
来源：omarsar0 · x.com