Jan Kautz | Chlience Paper Archive

Apr 08, 2026

BroRL: Scaling Reinforcement Learning via Broadened Exploration

BroRL 把 RLVR 的 scaling 轴从“继续训练更多 step”扩展到“每个 prompt 采样更多 rollout”：作者从 one step RLVR 的 correct token probability mass 分解出一个可能为负的 unsampled coupling term，并说明增大 rollout size $N$ 会让未采样项的二阶矩衰减，从而让 policy update 更稳定地增加正确 toke...

2510.01180-brorl-broadened-rl-exploration Rollout Optimization Reasoning RL RL Algorithm

Mar 10, 2026

ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models

这篇论文是对 “RLVR 只提升 base model 已有解的采样效率” 观点的直接反驳：作者提出 ProRL，用高温 rollout、DAPO 式 decoupled clipping/dynamic sampling、KL regularization、周期性 reference policy 与 optimizer reset，以及 136K 多任务 verifiable reward 数据，把 DeepSeek R1 Dis...

2505.24864-prorl-prolonged-rl-reasoning-boundaries Reasoning RL RL Algorithm Rollout Optimization

Related Notes

BroRL: Scaling Reinforcement Learning via Broadened Exploration

ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models