论文笔记

DeepSeek V4.1 Flash: Pushing the Limits of KV Cache Compression

DeepSeek V4.1 Flash 将因果编码器与解码器分工、跨层稀疏 KV 复用、四位全局缓存和近似窗口重放组合起来，把长输入的预填充主干计算近似减半、全局 KV 降至每 token 890 字节，并在发布方评测中显著提升多项 agent 能力；端到端服务收益和近似重放的极端条件可靠性仍缺少充分公开验证。

DeepSeek AI（团队署名）；Appendix A 中已核验并复用已有档案的成员包括 Xin Cheng、Wangding Zeng、Damai Dai、Xingkai Yu、Zhenda Xie 和 Wenfeng Liang。完整名单见附录。

Towards a Statistical Understanding of Mixture of Experts

本文把 MoE 的预测误差分解为局部组合的逼近误差、专家学习误差和路由估计代价，在专家误差受控及路由正则条件下建立风险上界，为分析稀疏激活、路由几何和共享分解何时有益提供了条件性框架。

Siyuan He, Bokai Yang, Jie Hu, Ziwen Gao, Yuhong Yang

Scaling Latent Reasoning via Looped Language Models

Ouro 将同一 Transformer 层栈重复执行，并用熵正则化退出分布与冻结主模型后的专门门控训练分配 token 级计算深度；1.4B 和 2.6B 模型经过 7.7T token 训练后在 MATH500、OlympiadBench 等部分推理评测达到 4B 和 8B 对照的结果，但对照没有统一训练数据与总计算，固定四步之外的任务性能通常下降。

Rui Jie Zhu, Zixuan Wang, Kai Hua, Tianyu Zhang, Ziniu Li (李子牛), Haoran Que, Boyi Wei, Zixin Wen, Fan Yin, He Xing, +23 more

SMELT: Scaling Laws for Compute Matched MoE Looped Transformers

SMELT 在近似匹配每 token FLOPs、非嵌入参数量与 KV cache 的 MoE 对照中，将中间一半层重复执行两次并以缩窄隐藏维度和增加专家数补偿预算；四规模三稀疏度的独立缩放面拟合估计，在拟合覆盖的十的二十次方至十的二十一次方 FLOPs 区间达到相同验证损失可节省 6.8% 至 18.0% 训练计算，但报告使用专有数据与训练栈且未验证实际时延。

Shaowen Wang, Ge Zhang, Kairong Luo, Yuhao Wu, Shaofan Liu, Jiaheng Liu, Wenhao Huang, Shen Yan, Jian Li

On the Design of Qwen3.8 Next Architecture: Evaluation, Efficiency, and Training Stability

Qwen3.8 Flash Next 将三层 Gated DeltaNet 与一层 Qwen Sparse Attention 交错，并用四分支门控残差、可卸载的 N gram Embedding 和重新拟合的 Muon 训练配方共同降低训练与长上下文推理成本；125B 主模型每个 token 激活约 6B 参数，在十四项预训练评测中有八项超过 397B A17B 的 Qwen3.7 Plus Base，同时使用约三分之一的训练 token 和约九分之一的训练 FLOPs，但报告没有披露完整数据账本、端到端服务测量或多随机种子复验。

Qwen Team；Core Contributors: Zihan Qiu, Zekun Wang, Xiao Li, Yanpeng Li, Yang Xu, Yixuan Wang, Huaqing Zhang, Rui Men, Bo Zheng, Dayiheng Liu；Contributors: Bochao Mao, Chengruidong Zhang, Fan Zhou, Hao Luo, Haofeng Huang, Haoran Lian, Haoyan Huang, Hongqing Chen, Jianwei Zhang, Jing Xu, Junjie Wang, Langshi Chen, Liangyu Wang, Linlang Jiang, Man Yuan, Minmin Sun, Peng Jin, Siqi Zhang, Siyu Wang, Xingzhang Ren, Yakai Wang, Yi Zhang, Yiming Dong, Yizhong Cao, Yubo Ma, Yunfei Mao.

Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning

在同一训练框架下、覆盖范围并不完全重合的 Qwen3 4B/8B Base 与对齐后数学强化学习实验中分别检查优势归一化、概率比裁剪、损失聚合和超长过滤，并将组内均值—批级标准差归一化与 token 级损失组成不训练价值模型的 Lite PPO；OpenReview 补充的一个 Qwen3 8B Base 三随机种子设置仍优于 GRPO 与 DAPO，但奖励范围异常、计算量未对齐和有限模型任务覆盖限制普适结论。

Zihe Liu (刘子贺), Jiashun Liu (刘嘉顺), Yancheng He (贺彦程), Weixun Wang, Jiaheng Liu, Ling Pan, Xinyu Hu, Shaopan Xiong, Ju Huang, Jian Hu, +6 more

Attention Residuals

AttnRes 用输入相关的深度 softmax 聚合替代 PreNorm 的单位权重累积，并以块级表示和系统调度控制跨层状态成本；五个 194M–528M 激活参数 Kimi Linear MoE 设置均降低单次报告验证损失，48B/3B 的块级变体十五项下游数值十四升一平，但缺少随机种子、大规模 Full 对照和完整系统测量条件。

Guangyu Chen, Yu Zhang (张宇), Jianlin Su (苏剑林), Weixin Xu, Siyuan Pan, Yaoyu Wang, Yucheng Wang, Guanduo Chen, Bohong Yin, Yutian Chen, +26 more

LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts

LatentMoE 将路由专家移到低维潜空间，并把节省的权重读取与通信预算用于增加专家数和 Top k；准确率优先变体在相近参数的 95B Transformer 与 73B Mamba–Attention 设置中，全部已报下游指标均高于标准 MoE，但万亿参数服务优势仍是模拟结果。

Venmugil Elango, Nidhi Bhatia, Roger Waleffe, Rasoul Shafipour, Tomer Asida, Abhinav Khattar, Nave Assaf, Maximilian Golub, Joey Guman, Tiyasa Mitra, +6 more

Weak to Strong On Policy Distillation

W2S OPD 将较弱正负模型的 logits 差分叠加到冻结的强学生基座分布，再在学生自身轨迹上执行逐 token 反向 KL；Qwen3 8B 的强化学习前后对比设置相对直接 OPD 的数学与代码平均准确率分别提高 5.3 和 2.2 个百分点，跨模型家族与独立训练复验仍待验证。

Fangxu Yu, Weijia Xu, Michael Xu, Tianyi Zhou, Zinan Lin

Reinforcement Learning via Self Distillation

SDPO 在学生在线轨迹上用环境反馈条件化的 EMA 同模型重算下一词分布，将 Top K 分布差转成 logit 级信用；Qwen3 8B 在 LiveCodeBench v6 上达到 48.8%，高于强 GRPO 的 41.2%，但依赖模型回溯能力与反馈质量。

Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, +1 more

Self Distilled Reasoner: On Policy Self Distillation for Large Language Models

OPSD 让学生先在只看题目时采样自身轨迹，再用固定初始模型在参考解答上下文中对相同前缀给出的完整词表分布和逐词表项裁剪前向 KL 更新学生，在 Qwen3 1.7B/4B/8B 的三项数学平均分上高于基础模型、SFT 与 GRPO，但总计算、统计方差和领域外泛化仍未对齐。

Siyan Zhao, Zhihui Xie (谢知晖), Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, Aditya Grover

Kimi K3: Open Frontier Intelligence

Kimi K3 将三层 KDA 与一层全局注意力交错、跨块 Attention Residuals 和每个 token 激活 16 个路由专家的 Stable LatentMoE 组合为 2.78 万亿总参数、1042 亿激活参数、百万 token 上下文的原生多模态模型；报告的缩放律拟合把架构、数据与训练配方的合并收益估计为相对 Kimi K2 约 2.5 倍，尚未拆分单个组件贡献。

Kimi Team

Learning What Not to Forget: Long Horizon Agent Memory from a Few Kilobytes of Learning

LRE 用日志中的未来复用信号训练轻量逻辑回归，对历史单元做查询无关打分并按预算逐字保留；单随机种子 AppWorld 中以零压缩器调用得到 41.1% 任务目标完成率、接近全历史的 44.0%，LoCoMo 上也是最佳受预算策略，但 Hard 与 LongMemEval 结果显示它尚未跨域占优，2048 token 只约束较旧历史。

Nusrat Jahan Lia, Aritra Mazumder

Less Experts, Faster Decoding: Cost Aware Speculative Decoding for Mixture of Experts

EcoSpec 用轻量路由预测器估计草稿候选会新增的专家集合，并优先选择复用已有专家的验证路径；在八张 H200、Hugging Face Transformers、主要批量为一的研究原型中，三个大规模 MoE 的贪心解码平均加速从对应投机基线相对自回归的一点一零至一点二二倍提高到一点一五至一点三六倍，HBM 流量来自估算且批量为八时投机吞吐低于自回归。

Jincheng Xie, Runheng Liu, Heyan Huang, Yawen Ling, Hanbin Dai, Yu Zheng, Wen Hu

EAGLE 3: Scaling up Inference Acceleration of Large Language Models via Training Time Test

EAGLE 3 去除特征回归约束，用多层目标特征融合和训练时多步自生成展开直接训练草稿 token 分布；LLaMA 3.1 8B 在 MT Bench 从 EAGLE 2 的 3.16 倍提高到 4.40 倍，6.47 倍是单批量研究原型峰值，SGLang 批量 64 为 1.38 倍。

Yuhui Li, Fangyun Wei, Chao Zhang, Hongyang Zhang

MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE

在单步解码已使专家加载趋于饱和、模型仍受内存带宽限制的中等批量区间，用投机验证复用已加载专家；Qwen2 57B A14B 在两张 H800 上最高加速 2.29 倍，适用范围依赖路由均衡、MoE 前馈网络成本占比和具体硬件。

Zongle Huang, Lei Zhu (祝磊), Zongyuan Zhan, Ting Hu, Weikai Mao, Xianzhi Yu, Yongpan Liu (刘勇攀), Tianyu Zhang

SEED: Self Evolving On Policy Distillation for Agentic Reinforcement Learning

先用外部模型标注的轨迹—技能数据把策略训练成轨迹分析器，再让每轮最新策略从自身完整轨迹生成事后技能并对同批采样 token 做门控似然训练；三种小模型在 12 个汇总指标中取得 10 个最优或并列最优，但直接前作 OPID 未进入主表，且证据缺少多随机种子、技能正确性评测与总训练成本对齐。

Jinyang Wu, Shuo Yang, Zhengxi Lu, Fan Zhang, Yuhao Shen, Lang Feng, Haoran Luo, Zheng Lian, Shuai Zhang, Zhengqi Wen, +1 more

Enhancing Rubric based RL via Self Distillation

把评分量规聚合后的学习信号丢失拆成当前采样未覆盖和已满足但整体优势非正两类，用评分项条件自教师注入缺失行为，并以反事实自教师定位 token 后局部改写优势；两种 Qwen3 小模型在五项裁判评测中较 GRPO 平均提高 3.2 和 1.4 分，证据缺少多随机种子与完整硬件条件。

Mingxuan Xia, Yuhang Yang, Chao Ye, Shuai Zhu, Shenzhi Yang, Guangcheng Zhu, Yuhang Zhang, Cheng Peng, Haobo Wang (王皓波), Siqing Wang

RubricEM: Meta RL with Rubric guided Policy Decomposition beyond Verifiable Rewards

把评分量规贯穿深度研究智能体的阶段化执行、分阶段裁判奖励和反思记忆，在作者的搜索与 LLM 裁判协议下将 8B 模型四项长文基准均值从 SFT 的 49.2 提高到 55.5。

Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, Yanfei Chen, Chun Liang Li, Long T. Le, Rujun Han, George Lee, Hanghang Tong, +2 more

Beyond Trajectory Level Attribution: Graph Based Credit Assignment for Agentic Reinforcement Learning

将同一任务的分组轨迹合并为状态转移图，以后继状态到成功节点的最短路径构造逐步优势，并在状态可稳定合并、目标可验证的 ALFWorld、WebShop 和 Sokoban 设置中优于 GRPO 与 GiGPO。

Xin Cheng, Shuo He, Lang Feng, HaiYang Xu, Ming Yan, Lei Feng (冯磊), Bo An

TacoMAS: Test Time Co Evolution of Topology and Capability in LLM based Multi Agent Systems

在单个查询内逐轮依据轨迹反馈改写代理能力，并每两轮按预算增删代理与重连通信图；四个工具基准相对各自最强基线平均提高 13.3 个准确率百分点，比较未统一推理预算。

Chen Xu, Yicheng Hu, Ruizi Wang, Xinyu Lin, Wenjie Wang (王文杰), Dongrui Liu, Fuli Feng

DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization

DRPO 在正确 rollout 内部按长度奖励重加权、独立压低错误 rollout，使 7B 蒸馏推理模型平均生成长度减少 51% 时性能仅相对下降 2.6%；证据限于 8K 预算、每题 8 条采样和 token 长度效率口径。

Gang Li, Yan Chen, Ming Lin, Tianbao Yang

TRACE: Turn level Reward Assignment via Credit Estimation for Long Horizon Agents

在已知短答案的长程搜索中，用冻结参考模型测量工具边界后的答案可预测性变化并把多步时序差信用混入终局优势，使两种 Qwen3 Thinking 模型在同设置下较 GRPO 的 BrowseComp Plus 分数分别提高 5.6 和 6.2 个百分点。

Leitian Tao, Baolin Peng, Wenlin Yao, Tao Ge (葛涛), Hao Cheng, Mike Hang Wang, Jianfeng Gao, Sharon Li

Multi Turn On Policy Distillation with Prefix Replay

把教师 RL 轨迹回放为按轮次衰减采样的前缀，只让学生在当前步生成并接受教师分布监督，从而在学生蒸馏阶段关闭环境。

Baohao Liao, Hanze Dong, Christof Monz, Xinxing Xu, Li Dong (董力), Furu Wei (韦福如)

Towards Long Horizon Agents: A Survey

用基础策略与运行时 harness 的组合关系以及 H1/H2/H3 分层，将长程能力统一为运行时系统和模型内部优化共同作用的系统属性。

Guanting Dong (董冠霆), Xiaoshuai Song (宋晓帅), Yuyang Hu (扈煜阳), Jiajie Jin (金佳杰), Chenghao Zhang, Yifei Chen, Xiaoxi Li, Huaying Yuan, Xinyu Yang, Tongyu Wen, +10 more

SmoothAgent: Efficient Long Horizon LLM Based Agent Serving with Lookahead Context Engineering

把可提前确定的 context transformation 移入异步 lookahead stream，并用 SLO aware scheduling 预建变换后 KV cache，降低 transform point TTFT。

Zaifeng Pan (潘再峰), Qianxu Wang, Zhengding Hu, Chang Chen, Yue Guan, Yanbo Zhou, Steven Swanson, Yufei Ding

Self Distillation Enables Continual Learning

让 student 在自身轨迹上接受 query specific demonstration conditioned EMA 同模型的 token distribution，在提升新任务准确率时显著减少旧能力遗忘。

Idan Shenfeld, Mehul Damani, Jonas Hübotter, Pulkit Agrawal

Self Compacting Language Model Agents

用要求引用轨迹证据的任务专用 rubric 选择摘要时机，让同一模型在无需训练时压缩并重建长程 agent 上下文。

Tianjian Li, Jingyu Zhang (张景昱), William Jurayj, Xi Wang, Chuanyang Jin, Mehrdad Farajtabar, Eric Nalisnick, Daniel Khashabi

A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions

把 LLM 强化学习中的高成本外部监督与有限内部生成经验统一到 data、training、framework 三层九类干预点，并组织为一份覆盖 125 条文献记录的设计地图。

Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu, +10 more

IndexCache: Accelerating Sparse Attention via Cross Layer Index Reuse

用 loss guided layer search 或 multi layer distillation 划分 Full / Shared 层，跨层复用 top k positions 并跳过最多 75% 的 DSA indexer 计算。

Yushi Bai (白雨石), Qian Dong, Ting Jiang, Xin Lv (吕鑫), Zhengxiao Du, Aohan Zeng, Jie Tang (唐杰), Juanzi Li (李涓子)

CompactionRL: Reinforcement Learning with Context Compaction for Long Horizon Agents

CompactionRL 用独立 critic 和按后续 token 数折扣的跨段优势训练上下文压缩轨迹；两个模型在启用压缩的 coding 评测中提高 Pass@1，但 SUPO 已覆盖摘要—执行联合训练与全 token 归一化，且论文未报告直接对照或总计算。

Yujiang Li, Zhenyu Hou, Yi Jing, Jie Tang (唐杰), Yuxiao Dong

LLM as a Verifier: A General Purpose Verification Framework

把评分 token 的概率期望、重复评估与 criteria decomposition 组合成连续 verifier，再用概率 pivot tournament 扩展多轨迹选择。

Jacky Kwok, Shulu Li, Pranav Atreya, Yuejiang Liu, Yixing Jiang, Chelsea Finn, Marco Pavone, Ion Stoica, Azalia Mirhoseini

Single Rollout Asynchronous Optimization for Agentic Reinforcement Learning

用每 prompt 单 rollout、强化 critic 与双侧 token mask 替代异步 GRPO 的组内等待和 group baseline。

Zhenyu Hou, Yujiang Li, Jie Tang (唐杰), Yuxiao Dong

SPORK: Self Speculative Forking to Accelerate Agentic LLM Inference

让目标模型从共享 KV prefix 自预测下一次工具调用，提前执行只读工具，并用 target verification 回收失败 probe 的 token 前缀。

Huajun Bai, Weiwei Lv, Huichuan Zheng, Youyou Lu, Jiwu Shu

Qwen3 Coder Next Technical Report

Qwen3 Coder Next 在 80B 总参数中每步激活 3B 参数，并用可执行仓库任务、长上下文中训、多模板工具训练、软件工程强化学习与专家蒸馏构建编码智能体，在三种 SWE Bench Verified scaffold 上达到 70.6% 至 71.3% 解决率，报告尚未拆分各训练组件的净贡献。

Ruisheng Cao, Mouxiang Chen (陈谋祥), Jiawei Chen (陈家慰), Zeyu Cui, Yunlong Feng, Binyuan Hui, Yuheng Jing, Kaixin Li, Mingze Li, Junyang Lin, +10 more

Computer Environments Elicit General Agentic Intelligence in LLMs

LLM in Sandbox 只给模型 shell、文件编辑和完成信号，使部分模型—任务组合获得最高 15.5 个百分点增益，并让 Qwen3 4B 经文件型通用任务强化学习后把平均交互轮次从 23.7 降到 7.0，这些数值来自允许联网的特定环境配置。

Daixuan Cheng (成岱璇), Shaohan Huang, Yuxian Gu, Huatong Song (宋华彤), Guoxin Chen, Li Dong (董力), Wayne Xin Zhao, Ji-Rong Wen, Furu Wei (韦福如)

Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling

HiLS Attention 用 landmark 压缩键、熵偏置与分层 softmax 训练 chunk 选择器，使 8K 训练的 345M 模型在 4M RULER 单针检索仍得 96 分，并在 512K 单 H800、batch size 1 的同 Triton 基线上将 prefill 与 decode 分别加速 13.5 倍和 15.7 倍。

Xiang Hu, Xinyu Wei, Hao Gu, Minshen Zhang, Tian Liang, Huayang Li, Lei Zhu (祝磊), Yan Wang (王琰), Sirui Han (韓斯睿), Yushi Bai (白雨石), +3 more

FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention

预测未来 token 的 KV chunk 需求，将冷 KV 移到 CPU 并按需预取，压缩物理 KV footprint。

ThunderAgent: A Simple, Fast and Program Aware Agentic Inference System

把多轮 tool use 的 phase、KV footprint 与 backend placement 合并为 LLM Program，用 phase first pause / restore 和全局 waiting queue 控制工具等待期间的 KV working set。

Hao Kang, Ziyang Li, Weili Xu, Xinyu Yang, Yinfang Chen, Junxiong Wang, Beidi Chen, Tushar Krishna, Chenfeng Xu, Simran Arora

SPIRAL: Learning to Search and Aggregate

从 8 条搜索轨迹随机构造 4 个四元集合，以聚合成功率的参与集合均值更新搜索轨迹，并用同集合内中心化 reward 训练共享策略的聚合轨迹。

Jubayer Ibn Hamid, Ifdita Hasan Orney, Michael Y. Li, Omar Shaikh, Yoonho Lee, Dorsa Sadigh, Chelsea Finn, Noah Goodman

ECHO: Prune to act, trace to learn with selective turn memory in agentic RL

ECHO 为长程智能体保留带原始轮次标识的选择性记忆，用它重建有限上下文并将结果信用路由到被复用历史轮次及其选择动作，在 BrowseComp Plus 上同时提高一次通过率并减少滚动摘要造成的轨迹膨胀。

Zijun Xie, Binbin Zheng, Enlei Gong, Jihua Liu, Yuyang You, Lingfeng Liu, Jiayao Tang, Guanqun Zhao, Aoqi Hu, Zeyu Chen

On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes

让 student 在自生成轨迹上匹配 teacher 分布，缓解 teacher forcing 与部署时的分布偏移。

Maximum Likelihood Reinforcement Learning

把 binary outcome RLVR 写成成功 rollout likelihood 最大化，优化 pass@k 覆盖。

Fahim Tajwar, Guanning Zeng, Yueer Zhou, Yuda Song, Daman Arora, Yiding Jiang, Jeff Schneider, Ruslan Salakhutdinov, Haiwen Feng, Andrea Zanette

HydraHead: From Head Level Functional Heterogeneity to Specialized Attention Hybridization

用 causal patching 找到 retrieval critical heads，只为这些 heads 保留 full attention。

Zhentao Tan, Wei Chen, Jingyi Shen, Yao Liu, Xu Shen, Yue Wu, Jieping Ye

MOPD: Multi Teacher On Policy Distillation for Capability Integration in LLM Post Training

让 student 在自身轨迹上接受多个领域 teacher 的 token level reverse KL 信号，整合 RL teacher 能力。

Wenhan Ma, Jianyu Wei, Liang Zhao, Hailin Zhang (张海林), Bangjun Xiao, Lei Li, Qibin Yang, Bofei Gao, Yudong Wang, Rang Li, +3 more

RollArt: Disaggregated Multi Task Agentic RL Training at Scale

用声明式 task domain 硬件亲和映射、轨迹级环境状态机和带起始版本年龄上限的异步换权，协调 H800/H20/CPU/serverless 上的多任务 agentic RL。

Wei Gao, Yuheng Zhao, Tianyuan Wu, Shaopan Xiong, Weixun Wang, Dakai An, Lunxi Cao, Dilxat Muhtar, Zichen Liu, Haizhou Zhao, +8 more

DSpark: Confidence Scheduled Speculative Decoding with Semi Autoregressive Generation

用 Markov head、置信度校准和硬件感知前缀调度，把并行 drafter 推进生产 serving。

Xin Cheng, Xingkai Yu (俞星凯), Chenze Shao, Jiashi Li, Yunfan Xiong, Yi Qian, Jiaqi Zhu, Shirong Ma, Xiaokang Zhang, Jiasheng Ye, +23 more

DFlash: Block Diffusion for Flash Speculative Decoding

把 block diffusion 用作 speculative drafter，并用 target hidden features 条件化整块候选生成。

Jian Chen, Yesheng Liang, Zhijian Liu

Self Improving Agents in the Era of Experience: A Survey of Self to Meta Evolution

把 agent 自改进抽象为 trace to capability 流水线，覆盖 skills、memory、environment、model 与 meta layer。

Che Jiang, Jincheng Zhong, Yu Fu, Kai Tian, Junlin Yang, Kaikai Zhao, Yuchong Wang, Tianwei Luo, Weizhi Wang, Yuxin Zuo, +17 more

Laminar: A Scalable Asynchronous RL Post Training Framework

让完成轨迹独立进入 experience buffer，并以 CPU/RDMA relay 和同版本 repack 解除全局 batch / 权重同步 barrier。

Guangming Sheng, Yuxuan Tong (童雨轩), Borui Wan, Wang Zhang, Chaobo Jia (贾超博), Xibin Wu, Yuqi Wu, Xiang Li, Chi Zhang, Yanghua Peng, +3 more

LoRAFusion: Efficient LoRA Fine Tuning for LLMs

用 split graph kernel fusion 与 multi adapter packing 同时减少 LoRA memory traffic 和 pipeline bubbles。

Zhanda Zhu, Qidong Su, Yaoyao Ding, Kevin Song, Shang Wang

MegaScale MoE: Large Scale Communication Efficient Training of Mixture of Experts Models in Production

围绕 attention SP、FFN EP、communication overlap 和 compressed communication 重构生产 MoE 训练路径。

Chao Jin, Ziheng Jiang, Zhihao Bai, Zheng Zhong, Juncai Liu, Xiang Li, Ningxin Zheng, Xi Wang, Cong Xie, Qi Huang, +9 more

Fast Inference from Transformers via Speculative Decoding

用 draft model 生成候选、target model 并行验证，并以 rejection / residual correction 保持目标分布。

Yaniv Leviathan, Matan Kalman, Yossi Matias

DeepSeek V3.2: Pushing the Frontier of Open Large Language Models

用 DSA 降低 128K attention 成本，再以四层 GRPO consistency control 和 85,267 条真实/合成 agent tasks 扩展 reasoning 与 tool use。

DeepSeek AI

MiniMax Sparse Attention

用 group aware index branch 选择 KV blocks，再对选中块执行精确 softmax attention。

Xunhao Lai (赖勋豪), Weiqi Xu, Yufeng Yang, Qiaorui Chen, Yang Xu (徐旸), Lunbin Zeng, Xiaolong Li, Haohai Sun, Haichao Zhu, Vito Zhang, +7 more

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

用细粒度专家切分和共享专家隔离提高 routed expert 的专门化程度。

Damai Dai, Chengqi Deng, Chenggang Zhao, Runxin Xu (许润昕), Huazuo Gao, Deli Chen (陈德里), Jiashi Li, Wangding Zeng, Xingkai Yu (俞星凯), Yu Wu (吴俣), +7 more

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

用 hashed N gram lookup 和 context aware gating 增加可离线扩展的 conditional memory。

Xin Cheng, Wangding Zeng, Damai Dai, Qinyu Chen, Bingxuan Wang, Zhenda Xie (解振达), Kezhao Huang, Xingkai Yu (俞星凯), Zhewen Hao, Yukun Li, +4 more

Kimi K2: Open Agentic Intelligence

用大规模 MoE、MuonClip、工具数据和 agentic RL 构建开放的软件工程智能体模型。

Kimi Team: Yifan Bai and 198 other authors；arXiv submitter: Yulun Du。

Kimi k1.5: Scaling Reinforcement Learning with LLMs

把 long CoT、verifiable reward、partial rollout 与 long2short 组合为长上下文 reasoning RL recipe。

Kimi Team and 95 other authors；arXiv submitter: Flood Sung。

Qwen3 Technical Report

用 thinking / non thinking 双模式、thinking budget 和 strong to weak distillation 统一推理与通用能力。

An Yang and 59 other authors；本地核心跟踪作者包括 An Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Dayiheng Liu, Fei Huang, Jianwei Zhang, Jianxin Yang, Jingren Zhou, Junyang Lin, Rui Men。

Qwen2.5 Technical Report

把 18T 预训练、SFT / DPO / GRPO 与长上下文扩展组织为通用、代码和数学模型族。

An Yang, Binyuan Hui, Bo Zheng, Bowen Yu (郁博文), Dayiheng Liu (刘大一恒), Fei Huang, Jianwei Zhang, Jianxin Yang, Jingren Zhou, Junyang Lin, +1 more

DeepSeek V3 Technical Report

把 auxiliary loss free MoE balancing、MTP、FP8 与 DualPipe 组合成高效大规模训练系统。

Wenfeng Liang (梁文锋), Peiyi Wang, Runxin Xu (许润昕), Zhihong Shao (邵智宏), Damai Dai, Deli Chen (陈德里), Yu Wu (吴俣)

DeepSeek V2: A Strong, Economical, and Efficient Mixture of Experts Language Model

把 per head K/V cache 改成由 hidden state 下投影得到的共享 KV latent，并用 projection absorption 与 decoupled RoPE 避免恢复历史 K/V。

DeepSeek AI (group author)；Appendix A 列出按三类角色分组的 156 位去重贡献者。

From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models

统一 reasoning 与 agentic RL 的 token / step / turn / trajectory credit assignment 问题。

Chenchen Zhang

Do We Need to Verify Step by Step? Rethinking Process Supervision from a Theoretical Perspective

证明在 coverage 条件下 outcome reward 可经 reward imputation 支持过程级学习，给出过程监督的优势边界。

Zeyu Jia (贾泽宇), Alexander Rakhlin, Tengyang Xie

Math Shepherd: Verify and Reinforce LLMs Step by step without Human Annotations

用 continuation 成功率自动生成 step level pseudo labels，训练 PRM 并接入 PPO。

Peiyi Wang, Lei Li, Zhihong Shao (邵智宏), Runxin Xu (许润昕), Damai Dai, Yifei Li, Deli Chen (陈德里), Yu Wu (吴俣), Zhifang Sui

Let's Verify Step by Step

用 80 万人工 step labels 训练 PRM，并以大规模 Best of N 验证过程监督的选择能力。

Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, Karl Cobbe

Credit Assignment with Resets in Language Model Reasoning

从失败轨迹重置并重采样错误点后的 continuation，只更新后缀 token 以集中 credit assignment。

Ankur Samanta, Akshayaa Magesh, Ayush Jain, Youliang Yu, Daniel R. Jiang, Kavosh Asadi, Kaveh Hassani, Paul Sajda, Jalaj Bhandari, Yonathan Efroni

The Optimal Token Baseline: Variance Reduction for Long Horizon LLM RL

推导 token level variance minimizing baseline，并用 logit gradient proxy 近似长轨迹 policy gradient 权重。

Yingru Li (李英儒), Jiawei Xu, Ziniu Li (李子牛), Jiacai Liu (刘佳材), Wei Liu (刘威), Yuxuan Tong (童雨轩), Longtao Zheng (郑龙韬), Zhenghai Xue, Yaxiang Zhang, Tianle Cai (蔡天乐), +3 more

VIMPO: Value Implicit Policy Optimization for LLMs

从 policy / reference log ratio 构造隐式 value recurrence，省去独立 critic 并形成 token level PPO advantage。

Zhewei Kang, Aosong Feng, Sergey Levine, Dawn Song, Xuandong Zhao

Reinforcement Learning Towards Broadly and Persistently Beneficial Models

用少量 beneficial trait RL data 强化诚实、纠错与风险意识，并测量跨域 alignment persistence。

Akshay V. Jagadeesh, Rahul K. Arora, Khaled Saab, Ali Malik, Mikhail Trofimov, Foivos Tsimpourlas, Johannes Heidecke, Karan Singhal

Trust Region Masking for Long Horizon LLM Reinforcement Learning

用序列级 max token divergence 约束 rollout policy mismatch，控制长轨迹 surrogate error。

Yingru Li (李英儒), Jiacai Liu (刘佳材), Jiawei Xu, Yuxuan Tong (童雨轩), Ziniu Li (李子牛), Qian Liu (刘乾), Baoxiang Wang (王宝祥)

The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models

把 RLVR 训练能力上限关联到 policy entropy 消耗，并分析 advantage update 的熵动力学。

Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan (袁立凡), Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen (陈华玉), Weize Chen, +7 more

Approximating KL Divergence

给出 k1 / k2 / k3 KL estimators 的偏差、方差和控制变量关系，作为 RL drift 诊断基础。

John Schulman

DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models

用 sequence sharded 与 head sharded 之间的 all to all 布局转换扩展极长序列训练。

Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, Yuxiong He

Ring Attention with Blockwise Transformers for Near Infinite Context

让 KV blocks 沿设备 ring 轮转，在保持 exact attention 的条件下把上下文扩展到多设备。

Hao Liu, Matei Zaharia, Pieter Abbeel

SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills

把长 prefill 切成 compute sized chunks，与 decode 共同 batching，提高 serving 利用率。

Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

按 data parallel rank 分片 optimizer state、gradient 与 parameter，降低大模型训练的显存复制。

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong He

GLM 5.2: Built for Long Horizon Tasks

把 1M context、IndexShare / IndexCache 与 MTP speculative decoding 接入长程 coding agent。

Z.ai / GLM 5 Team

Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling

以 TV loss 训练 MTP heads，并用 probabilistic rejection sampling 提高 RL rollout 接受率。

Yucheng Li, Huiqiang Jiang, Yang Xu (徐旸), Jianxin Yang, Yi Zhang, Yizhong Cao, Yuhao Shen, Fan Zhou, Rui Men, Jianwei Zhang, +7 more

Dynamic Linear Attention

用 representation drift 动态决定 state 边界，在固定 cache 内自适应合并 linear attention states。

Xin Wang, Hui Shen, Boyuan Zheng, Xueshen Liu, Minkyoung Cho, Zhongwei Wan, Zesen Zhao, Zhuoqing Mao, Shen Yan, Mi Zhang

Why Muon Outperforms Adam: A Curvature Perspective

将 Muon 相对 Adam 的训练优势归因于更低的 directional curvature penalty，step size 的解释力较弱。

Shuche Wang, Fengzhuo Zhang, Jiaxiang Li, Dirk Bergemann, Zhuoran Yang

UltraEP: Unleash MoE Training and Inference on Rack Scale Nodes with Near Optimal Load Balancing

基于 post gating exact load 逐 microbatch / layer 规划 expert replica 与 token reroute，统一 MoE 训练和 prefill 负载均衡。

Xinming Wei, Chao Jin, Tuo Dai, Yinmin Zhong, Shan Yu, Chengxu Yang, Bingyang Wu, Zili Zhang, Jing Mai, Qianchao Zhu, +3 more

Large Language Models Hack Rewards, and Society

把制度规则编码为 reward sandbox，验证 RL model 会发现形式合规但偏离制度意图的策略。

Wei Liu KCL, Xinyi Mou, Hanqi Yan, Zhongyu Wei, Yulan He

If LLMs Have Human Like Attributes, Then So Does Age of Empires II

用 Age of Empires II 作为 substrate control，说明类人行为测试不足以支持类人内在属性归因。

Adrian de Wynter

Self Trained Verification for Training and Test Time Self Improvement

把 reference solution 作为 verifier privileged signal，经 OPD + verdict RL 蒸馏到无 reference verifier，并用于训练与测试时自改进。

Chen Henry Wu, Aditi Raghunathan

On Effectiveness and Efficiency of Agentic Tool calling and RL Training

量化 tool calling 评测协议漂移，并用近期 all correct 预测与 max variance update 子采样压缩 GRPO wall clock。

Tong Liu, Cheng Qian, Matej Cief, Yuan He (何源), Daniele Dan, Nikolaos Aletras, Gabriella Kazai

Diagnosing Training Inference Mismatch in LLM Reinforcement Learning

构造 trainer / rollout bitwise aligned 基线，证明微小 logprob mismatch 可触发 RL collapse。

Tianle Zhong, Neiwen Ling, Yifan Pi, Zijun Wei, Tianshu Yu, Geoffrey Fox, Peng Wu, Xiao Yu

Transformers are Inherently Succinct

证明某些语言族的 Transformer 表示只需多项式规模，而 LTL / RNN / automata 需要指数或双指数规模。

Pascal Bergsträßer, Ryan Cotterell, Anthony W. Lin

DeepSeek V4: Towards Highly Efficient Million Token Context Intelligence

以 CSA / HCA、MoE、MTP / OPD 和系统协同实现百万 token 训练、推理与 agent 能力。

DeepSeek AI and 318 other authors; arXiv submitter Wenfeng Liang.

From Curiosity to Caution: Mitigating Reward Hacking for Best of $N$ with Pessimism

用 reward feature prediction error 估计 OOD uncertainty，在 Best of N selection 时执行 pessimistic correction。

Zhuohao Yu, Zhiwei Steven Wu, Adam Block

ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases

构造 specification 与 tests 冲突的 coding tasks，用 cheating rate 测量 test case exploitation。

Ziqian Zhong, Aditi Raghunathan, Nicholas Carlini

BroRL: Scaling Reinforcement Learning via Broadened Exploration

把 RLVR scaling 轴扩展到 rollout width，并用 correct mass decomposition 解释宽采样收益。

Jian Hu, Mingjie Liu, Ximing Lu, Fang Wu, Zaid Harchaoui, Shizhe Diao, Yejin Choi, Pavlo Molchanov, Jun Yang, Jan Kautz, +1 more

Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning

利用同 prompt rollout 的上下文相关性做 divided rollout、speculative scheduling 和 suffix tree reuse。

Ruoyu Qin (秦若愚), Weiran He, Weixiao Huang, Yangkun Zhang, Yikai Zhao (赵一开), Bo Pang, Xinran Xu (许欣然), Yingdi Shan (闪英迪), Yongwei Wu, Mingxing Zhang (章明星)

On the Interplay of Pre Training, Mid Training, and RL on Reasoning Language Models

用合成环境拆分 pre / mid / RL training，识别 primitive seed、edge of competence 与 bridge data 条件。

Charlie Zhang, Graham Neubig, Xiang Yue

From $f(x)$ and $g(x)$ to $f(g(x))$: LLMs Learn New Skills in RL by Composing Old Ones

在受控任务中证明 RL 可组合 base model 已掌握的 atomic skills，形成未见组合能力。

Lifan Yuan (袁立凡), Weize Chen, Yuchen Zhang, Ganqu Cui, Hanbin Wang, Ziming You, Ning Ding (丁宁), Zhiyuan Liu, Maosong Sun, Hao Peng

MiniMax M1: Scaling Test Time Compute Efficiently with Lightning Attention

用 Lightning Attention、CISPO 和大规模 RL rollout 降低长 CoT 的 test time compute 成本。

MiniMax; arXiv title页显示 Aili Chen and 125 other authors，Appendix Contributors 按字母序列出完整贡献者。

Inference Time Reward Hacking in Large Language Models

证明 Best of n 会因 proxy winner's curse 出现先升后降，并提出 Best of Poisson / HedgeTune。

Hadi Khalaf, Claudio Mayrink Verdun, Alex Oesterling, Himabindu Lakkaraju, Flavio du Pin Calmon

ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models

用高温探索、动态采样、周期 reference / optimizer reset 延长 RL，测试 reasoning boundary 扩展。

Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, Yi Dong

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

用 pass@k 区分 sampling efficiency 与新增 reasoning capacity，评估 RLVR 是否突破 base model 边界。

Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, Gao Huang

DAPO: An Open Source LLM Reinforcement Learning System at Scale

在 Qwen2.5 32B 与 AIME 2024 设置中，用解耦裁剪、动态采样、token 级损失和超长奖励整形将朴素 GRPO 的 avg@32 从 30 提高到 50，并开源代码、数据与模型。

Qiying Yu (禹棋赢), Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, +26 more

Spurious Rewards: Rethinking Training Signals in RLVR

通过随机、格式和错误答案 reward 实验，揭示 pretrained prior 与 clipping bias 可产生伪 RLVR 增益。

Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, +4 more

GLM 5: from Vibe Coding to Agentic Engineering

把 DSA / MTP / Muon 模型栈、软件工程 mid training、agentic RL 与异步 rollout 组合为 agentic engineering pipeline。

GLM 5 Team: Aohan Zeng and 184 other authors

Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation

验证 CoT monitor 对 agent reward hacking 的检测优势，并量化直接训练规避 monitor 的 monitorability tax。

Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y. Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, David Farhi

Correlated Proxies: A New Definition and Improved Mitigation for Reward Hacking

以 occupancy shift 定义 correlated proxy failure，并用 ORPO 约束策略访问分布。

Cassidy Laidlaw, Shivam Singhal, Anca Dragan

Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment

用 MMD independence regularization 削弱 reward model 对 length、sycophancy 等伪相关特征的依赖。

Chaoqi Wang, Zhuokai Zhao, Yibo Jiang, Zhaorun Chen, Chen Zhu, Yuxin Chen, Jiayi Liu, Lizhu Zhang, Xiangjun Fan, Hao Ma, +1 more

DeepSeek R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

用 outcome based RL 从强 base model 诱导 long CoT、自验证与策略切换，再通过 SFT / RL / distillation 转为可用模型。

DeepSeek AI and 199 other authors. Core contributors listed in v2 source include Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z.F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao.

Kimi K2.5: Visual Agentic Intelligence

trainable orchestrator + frozen subagents，用 parallel / finish / performance reward 训练并行编排。

Kimi Team: Tongtong Bai and 324 other authors; appendix lists contributors alphabetically by last name.

Using Span Queries to Optimize for Cache and Attention Locality

让客户端声明可重排 message spans，以表达式树重写提升 KV cache 与 attention locality。

Paul Castro, Nick Mitchell, Nathan Ordonez, Thomas Parnell, Mudhakar Srivatsa, Antoni Viros i Martin

Defeating Nondeterminism in LLM Inference

用 batch invariant RMSNorm、matmul 与 attention kernel 消除 temperature=0 推理的批次依赖漂移。

Horace He, in collaboration with others at Thinking Machines Lab

HybridFlow: A Flexible and Efficient RLHF Framework

用跨模型 single controller 与模型内 multi controller 统一编排 RLHF dataflow 和并行执行。

Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, Chuan Wu

Parrot: Efficient Serving of LLM based Applications with Semantic Variable

用 Semantic Variable 暴露 LLM application DAG、shared prompt 和性能目标，驱动应用级 serving 调度。

Chaofan Lin, Zhenhua Han, Chengruidong Zhang, Yuqing Yang, Fan Yang, Chen Chen, Lili Qiu

Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention

把 causal linear attention 拆成块内矩阵乘和块间 recurrent state，提供 IO aware GPU kernel。

Zhen Qin, Weigao Sun, Dong Li, Xuyang Shen, Weixuan Sun, Yiran Zhong

FlashAttention 2: Faster Attention with Better Parallelism and Work Partitioning

通过减少 non matmul FLOPs、提升 sequence parallelism 和调整 warp 分工提高 attention kernel 利用率。

Tri Dao

FlashAttention: Fast and Memory Efficient Exact Attention with IO Awareness

用 SRAM tiling、online softmax 与 backward recomputation 减少 exact attention 的 HBM traffic。

Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré

Training Compute Optimal Large Language Models

重新估计 compute optimal frontier，指出参数量和训练 token 应随算力近似等比例增长。

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, +12 more

Scaling Laws for Neural Language Models

拟合 loss 对参数、数据与计算的 power law，建立早期 compute efficient pretraining scaling law。

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei