NVIDIA 论文分享面向智能体编码的开源中训练配方
NVIDIA 发布论文，分享面向智能体编码的开源中训练配方：保持 SFT 不变、在预训练与后训练之间新增一个中训练阶段，在 256K 上下文上用 500B token 的自然软件工程数据（打包仓库、pull request、提交历史、构建日志及少量智能体轨迹）训练。
Great paper from NVIDIA sharing an open midtraining recipe for agentic coding.
NVIDIA kept SFT identical and added one midtraining stage. SWE-bench Multilingual went from 55.8 to 62.8 on Nemotron 3.5 Lightning.
The stage sits between pretraining and post-training. It trains at 256K context on 500B tokens of natural software-engineering data, such as packed repositories, pull requests, commit histories and build logs, plus a small share of agent traces.
On the same model, SWE-bench Verified rose from 60.7 to 64.5 and SWE-bench Pro from 32.0 to 35.5. On the larger Nemotron 3.5 Super, DeepSWE rose from 11.7 to 15.7.
They also explain how they evaluated the recipe. Base-model proxy metrics did not predict post-training results, so every data choice was scored with short fixed SFT runs.
If you train coding agents, the recipe and its ablations are published.
arxiv.org/abs/2610.22019
Chat with Paper: academy.dair.ai/papers/primi…
来源：dair_ai · x.com