natolambert· @natolambert · X·· 4 小时前AI 评分
GRPO 崛起后 RL 重要性采样成研究重点
我正在读分数中心化 RL 论文，想给我的书补充一些相关笔记。他们在附录中有一张很棒的表格，介绍了 RL 中各种基于重要性采样的训练-推理不匹配方法。在我看来，自 GRPO 起飞以来，这些一直是 RL 算法研究的最大焦点。
Was reading the score-centering RL paper to add some notes on it the my book, and they have a great table in the appendix on the different importance sampling based train-inference mismatch methods in RL. These have been the biggest focus of RL algorithm research since the GRPO takeoff, imo.
来源：natolambert · x.com