PostTrainBench 研究揭示智能体后训练中的策略锁定现象

An agent that can run a full post-training pipeline almost never reconsiders its initial strategy.

We analyze 1,338 complete training trajectories from PostTrainBench — 7 benchmarks, 4 base models, Claude Code / Codex CLI / OpenCode, 10h on one H100 — and split capability into EXECUTION vs STRATEGY.

1️⃣ Qualified executor, locked strategy. Agents improve over base models on every benchmark; with experience, avg score goes 33.34 → 48.55. ALL execution-level suggestions were adopted, but NONE of the strategy-level suggestions (e.g., switching to RL, adding SFT warmup) were adopted. Only 2.1% of 3,557 adjacent experiment pairs actually switch strategy. Defaults follow the agent, not the task: Claude Code starts full-parameter SFT 71.9% of the time; Codex CLI starts PEFT/LoRA 89.5%.

2️⃣ Experience and compute don't unlock it. Extra experience and memory raise execution but never leave the locked strategy; more reasoning tokens buy finer execution rather than better direction — on AIME 2025, ~14M extra tokens solve just 1 more problem.

3️⃣ A mid-run human guidance wins everywhere. At the same checkpoint, same remaining budget, one sentence switching to the alternative strategy beats autonomous continuation: GSM8K +17.44pp, HumanEval +10.80pp, AIME 2025 +6.67pp.

The missing ability isn't executing a new strategy — it's the trigger to spontaneously restart strategy decisions mid-trajectory.

📄 Paper: arxiv.org/abs/2608.19072

#AIAgent #PostTraining #LLM #RLHF #GRPO #PPO #RSI #AgentBenchmark #PostTrainBench #THUNLP