Research topics
研究主题
42 个细粒度主题按核心贡献组织，同一篇论文可以连接多个直接相关的问题域。
13 routes · 71 notes
Training
模型训练、后训练、优化信号与训练基础设施。
Distributed Training
10训练并行、通信、调度与多设备扩展。
RollArt: Disaggregated Multi Task Agentic RL Training at ScaleLaminar: A Scalable Asynchronous RL Post Training FrameworkLoRAFusion: Efficient LoRA Fine Tuning for LLMsMegaScale MoE: Large Scale Communication Efficient Training of Mixture of Experts Models in ProductionDeepSeek V3 Technical ReportDeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer ModelsRing Attention with Blockwise Transformers for Near Infinite ContextZeRO: Memory Optimizations Toward Training Trillion Parameter ModelsUltraEP: Unleash MoE Training and Inference on Rack Scale Nodes with Near Optimal Load BalancingHybridFlow: A Flexible and Efficient RLHF Framework
Training Memory
3训练状态、梯度、参数或 activation 的显存管理。
Parameter-Efficient Finetuning
1LoRA 等参数高效微调方法与系统。
Optimizer
3参数更新规则、曲率、缩放与优化器工程。
RL Algorithm
20直接修改 policy objective、advantage 或 trust region 的 RL 方法。
SEED: Self Evolving On Policy Distillation for Agentic Reinforcement LearningEnhancing Rubric based RL via Self DistillationBeyond Trajectory Level Attribution: Graph Based Credit Assignment for Agentic Reinforcement LearningDRPO: Efficient Reasoning via Decoupled Reward Policy OptimizationTRACE: Turn level Reward Assignment via Credit Estimation for Long Horizon AgentsCompactionRL: Reinforcement Learning with Context Compaction for Long Horizon AgentsSingle Rollout Asynchronous Optimization for Agentic Reinforcement LearningSPIRAL: Learning to Search and AggregateOn-Policy Distillation of Language Models: Learning from Self-Generated MistakesMaximum Likelihood Reinforcement LearningMOPD: Multi Teacher On Policy Distillation for Capability Integration in LLM Post TrainingCredit Assignment with Resets in Language Model ReasoningThe Optimal Token Baseline: Variance Reduction for Long Horizon LLM RLVIMPO: Value Implicit Policy Optimization for LLMsTrust Region Masking for Long Horizon LLM Reinforcement LearningBroRL: Scaling Reinforcement Learning via Broadened ExplorationMiniMax M1: Scaling Test Time Compute Efficiently with Lightning AttentionProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language ModelsDAPO: An Open Source LLM Reinforcement Learning System at ScaleDeepSeek R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Reasoning RL
24面向数学、代码与长推理能力的强化学习。
DRPO: Efficient Reasoning via Decoupled Reward Policy OptimizationA Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and SolutionsMaximum Likelihood Reinforcement LearningMOPD: Multi Teacher On Policy Distillation for Capability Integration in LLM Post TrainingKimi k1.5: Scaling Reinforcement Learning with LLMsQwen3 Technical ReportQwen2.5 Technical ReportFrom Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language ModelsMath Shepherd: Verify and Reinforce LLMs Step by step without Human AnnotationsCredit Assignment with Resets in Language Model ReasoningThe Optimal Token Baseline: Variance Reduction for Long Horizon LLM RLVIMPO: Value Implicit Policy Optimization for LLMsThe Entropy Mechanism of Reinforcement Learning for Reasoning Language ModelsSelf Trained Verification for Training and Test Time Self ImprovementBroRL: Scaling Reinforcement Learning via Broadened ExplorationSeer: Online Context Learning for Fast Synchronous LLM Reinforcement LearningOn the Interplay of Pre Training, Mid Training, and RL on Reasoning Language ModelsFrom $f(x)$ and $g(x)$ to $f(g(x))$: LLMs Learn New Skills in RL by Composing Old OnesMiniMax M1: Scaling Test Time Compute Efficiently with Lightning AttentionProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language ModelsDoes Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?DAPO: An Open Source LLM Reinforcement Learning System at ScaleSpurious Rewards: Rethinking Training Signals in RLVRDeepSeek R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Agent RL
22面向多轮环境交互、工具调用与 agent 行为的强化学习。
SEED: Self Evolving On Policy Distillation for Agentic Reinforcement LearningRubricEM: Meta RL with Rubric guided Policy Decomposition beyond Verifiable RewardsBeyond Trajectory Level Attribution: Graph Based Credit Assignment for Agentic Reinforcement LearningTRACE: Turn level Reward Assignment via Credit Estimation for Long Horizon AgentsMulti Turn On Policy Distillation with Prefix ReplayTowards Long Horizon Agents: A SurveyA Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and SolutionsCompactionRL: Reinforcement Learning with Context Compaction for Long Horizon AgentsSingle Rollout Asynchronous Optimization for Agentic Reinforcement LearningQwen3 Coder Next Technical ReportComputer Environments Elicit General Agentic Intelligence in LLMsECHO: Prune to act, trace to learn with selective turn memory in agentic RLRollArt: Disaggregated Multi Task Agentic RL Training at ScaleSelf Improving Agents in the Era of Experience: A Survey of Self to Meta EvolutionDeepSeek V3.2: Pushing the Frontier of Open Large Language ModelsKimi K2: Open Agentic IntelligenceFrom Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language ModelsReinforcement Learning Towards Broadly and Persistently Beneficial ModelsLarge Language Models Hack Rewards, and SocietyOn Effectiveness and Efficiency of Agentic Tool calling and RL TrainingGLM 5: from Vibe Coding to Agentic EngineeringKimi K2.5: Visual Agentic Intelligence
Credit Assignment
14把 trajectory reward 分配到 step、token、trace 或 agent。
SEED: Self Evolving On Policy Distillation for Agentic Reinforcement LearningEnhancing Rubric based RL via Self DistillationRubricEM: Meta RL with Rubric guided Policy Decomposition beyond Verifiable RewardsBeyond Trajectory Level Attribution: Graph Based Credit Assignment for Agentic Reinforcement LearningTacoMAS: Test Time Co Evolution of Topology and Capability in LLM based Multi Agent SystemsTRACE: Turn level Reward Assignment via Credit Estimation for Long Horizon AgentsCompactionRL: Reinforcement Learning with Context Compaction for Long Horizon AgentsSingle Rollout Asynchronous Optimization for Agentic Reinforcement LearningSPIRAL: Learning to Search and AggregateFrom Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language ModelsDo We Need to Verify Step by Step? Rethinking Process Supervision from a Theoretical PerspectiveCredit Assignment with Resets in Language Model ReasoningThe Optimal Token Baseline: Variance Reduction for Long Horizon LLM RLVIMPO: Value Implicit Policy Optimization for LLMs
Reward Modeling
16训练或构造 reward、proxy、preference 与可验证信号。
Enhancing Rubric based RL via Self DistillationRubricEM: Meta RL with Rubric guided Policy Decomposition beyond Verifiable RewardsDRPO: Efficient Reasoning via Decoupled Reward Policy OptimizationTRACE: Turn level Reward Assignment via Credit Estimation for Long Horizon AgentsA Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and SolutionsLLM as a Verifier: A General Purpose Verification FrameworkQwen2.5 Technical ReportMath Shepherd: Verify and Reinforce LLMs Step by step without Human AnnotationsLet's Verify Step by StepReinforcement Learning Towards Broadly and Persistently Beneficial ModelsFrom Curiosity to Caution: Mitigating Reward Hacking for Best of $N$ with PessimismDAPO: An Open Source LLM Reinforcement Learning System at ScaleSpurious Rewards: Rethinking Training Signals in RLVRCorrelated Proxies: A New Definition and Improved Mitigation for Reward HackingBeyond Reward Hacking: Causal Rewards for Large Language Model AlignmentDeepSeek R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Training Stability
11控制 entropy、分布偏移、数值误差与异步陈旧性。
Self Distillation Enables Continual LearningLaminar: A Scalable Asynchronous RL Post Training FrameworkDeepSeek V3.2: Pushing the Frontier of Open Large Language ModelsDeepSeek V3 Technical ReportTrust Region Masking for Long Horizon LLM Reinforcement LearningThe Entropy Mechanism of Reinforcement Learning for Reasoning Language ModelsApproximating KL DivergenceBreaking Entropy Bounds: Accelerating RL Training via MTP with Rejection SamplingWhy Muon Outperforms Adam: A Curvature PerspectiveDiagnosing Training Inference Mismatch in LLM Reinforcement LearningProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
On-Policy Distillation
9在 student 自生成轨迹上使用 teacher 分布或反馈。
SEED: Self Evolving On Policy Distillation for Agentic Reinforcement LearningEnhancing Rubric based RL via Self DistillationMulti Turn On Policy Distillation with Prefix ReplaySelf Distillation Enables Continual LearningOn-Policy Distillation of Language Models: Learning from Self-Generated MistakesMOPD: Multi Teacher On Policy Distillation for Capability Integration in LLM Post TrainingQwen3 Technical ReportSelf Trained Verification for Training and Test Time Self ImprovementDeepSeek R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
RL Infrastructure
7组织 actor、rollout、reward、trainer 与权重同步的训练系统。
RollArt: Disaggregated Multi Task Agentic RL Training at ScaleLaminar: A Scalable Asynchronous RL Post Training FrameworkDiagnosing Training Inference Mismatch in LLM Reinforcement LearningSeer: Online Context Learning for Fast Synchronous LLM Reinforcement LearningDAPO: An Open Source LLM Reinforcement Learning System at ScaleDefeating Nondeterminism in LLM InferenceHybridFlow: A Flexible and Efficient RLHF Framework
Rollout Optimization
15降低 rollout 等待、生成成本、长尾与采样浪费。
Multi Turn On Policy Distillation with Prefix ReplayA Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and SolutionsSingle Rollout Asynchronous Optimization for Agentic Reinforcement LearningRollArt: Disaggregated Multi Task Agentic RL Training at ScaleLaminar: A Scalable Asynchronous RL Post Training FrameworkKimi k1.5: Scaling Reinforcement Learning with LLMsCredit Assignment with Resets in Language Model ReasoningTrust Region Masking for Long Horizon LLM Reinforcement LearningBreaking Entropy Bounds: Accelerating RL Training via MTP with Rejection SamplingOn Effectiveness and Efficiency of Agentic Tool calling and RL TrainingDiagnosing Training Inference Mismatch in LLM Reinforcement LearningBroRL: Scaling Reinforcement Learning via Broadened ExplorationSeer: Online Context Learning for Fast Synchronous LLM Reinforcement LearningProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language ModelsHybridFlow: A Flexible and Efficient RLHF Framework
6 routes · 30 notes
Inference
在线推理、请求调度、缓存与 test-time compute。
Inference Scheduling
6请求、prefill、decode 或微任务的执行次序与 batch 构造。
SmoothAgent: Efficient Long Horizon LLM Based Agent Serving with Lookahead Context EngineeringEfficient Serving for Agentic LLM Workflows via Micro Task Level ParallelismThunderAgent: A Simple, Fast and Program Aware Agentic Inference SystemSARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked PrefillsUltraEP: Unleash MoE Training and Inference on Rack Scale Nodes with Near Optimal Load BalancingParrot: Efficient Serving of LLM based Applications with Semantic Variable
Serving Runtime
9面向在线负载的完整推理 runtime 与执行接口。
EAGLE 3: Scaling up Inference Acceleration of Large Language Models via Training Time TestMoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoESmoothAgent: Efficient Long Horizon LLM Based Agent Serving with Lookahead Context EngineeringIndexCache: Accelerating Sparse Attention via Cross Layer Index ReuseEfficient Serving for Agentic LLM Workflows via Micro Task Level ParallelismDSpark: Confidence Scheduled Speculative Decoding with Semi Autoregressive GenerationSARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked PrefillsUsing Span Queries to Optimize for Cache and Attention LocalityParrot: Efficient Serving of LLM based Applications with Semantic Variable
KV Cache
9KV 的存储、复用、迁移、抢占与容量管理。
SmoothAgent: Efficient Long Horizon LLM Based Agent Serving with Lookahead Context EngineeringEfficient Serving for Agentic LLM Workflows via Micro Task Level ParallelismSPORK: Self Speculative Forking to Accelerate Agentic LLM InferenceFlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse AttentionThunderAgent: A Simple, Fast and Program Aware Agentic Inference SystemDeepSeek V2: A Strong, Economical, and Efficient Mixture of Experts Language ModelSARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked PrefillsUsing Span Queries to Optimize for Cache and Attention LocalityParrot: Efficient Serving of LLM based Applications with Semantic Variable
Speculative Decoding
9通过 draft、验证与接受策略减少 target decode 次数。
EAGLE 3: Scaling up Inference Acceleration of Large Language Models via Training Time TestMoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoESPORK: Self Speculative Forking to Accelerate Agentic LLM InferenceDSpark: Confidence Scheduled Speculative Decoding with Semi Autoregressive GenerationDFlash: Block Diffusion for Flash Speculative DecodingFast Inference from Transformers via Speculative DecodingGLM 5.2: Built for Long Horizon TasksBreaking Entropy Bounds: Accelerating RL Training via MTP with Rejection SamplingSeer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning
Test-Time Scaling
10用多轨迹、搜索、验证或聚合扩大推理计算。
TacoMAS: Test Time Co Evolution of Topology and Capability in LLM based Multi Agent SystemsSelf Compacting Language Model AgentsLLM as a Verifier: A General Purpose Verification FrameworkSPIRAL: Learning to Search and AggregateLet's Verify Step by StepSelf Trained Verification for Training and Test Time Self ImprovementFrom Curiosity to Caution: Mitigating Reward Hacking for Best of $N$ with PessimismMiniMax M1: Scaling Test Time Compute Efficiently with Lightning AttentionInference Time Reward Hacking in Large Language ModelsUsing Span Queries to Optimize for Cache and Attention Locality
Deterministic Inference
2控制 kernel、batch 与浮点路径引入的输出差异。
9 routes · 36 notes
Architecture
模型结构、attention、MoE 与长上下文机制。
Long Context
18扩大训练或推理上下文并控制时间、显存与质量。
Self Compacting Language Model AgentsIndexCache: Accelerating Sparse Attention via Cross Layer Index ReuseHierarchical Sparse Attention Done Right: Toward Infinite Context ModelingFlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse AttentionECHO: Prune to act, trace to learn with selective turn memory in agentic RLHydraHead: From Head Level Functional Heterogeneity to Specialized Attention HybridizationMiniMax Sparse AttentionKimi k1.5: Scaling Reinforcement Learning with LLMsQwen3 Technical ReportQwen2.5 Technical ReportDeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer ModelsRing Attention with Blockwise Transformers for Near Infinite ContextGLM 5.2: Built for Long Horizon TasksDynamic Linear AttentionDeepSeek V4: Towards Highly Efficient Million Token Context IntelligenceGLM 5: from Vibe Coding to Agentic EngineeringVarious Lengths, Constant Speed: Efficient Language Modeling with Lightning AttentionFlashAttention: Fast and Memory Efficient Exact Attention with IO Awareness
Sparse Attention
7只访问被选择的 token、block 或 chunk。
IndexCache: Accelerating Sparse Attention via Cross Layer Index ReuseHierarchical Sparse Attention Done Right: Toward Infinite Context ModelingFlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse AttentionDeepSeek V3.2: Pushing the Frontier of Open Large Language ModelsMiniMax Sparse AttentionGLM 5.2: Built for Long Horizon TasksDeepSeek V4: Towards Highly Efficient Million Token Context Intelligence
Linear Attention
4用有限状态或核化计算替代完整二次 attention。
Attention Kernel
6attention kernel 的 IO、tiling、并行与硬件映射。
DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer ModelsRing Attention with Blockwise Transformers for Near Infinite ContextDefeating Nondeterminism in LLM InferenceVarious Lengths, Constant Speed: Efficient Language Modeling with Lightning AttentionFlashAttention 2: Faster Attention with Better Parallelism and Work PartitioningFlashAttention: Fast and Memory Efficient Exact Attention with IO Awareness
MoE Architecture
11专家划分、路由、共享专家与稀疏激活结构。
MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoEQwen3 Coder Next Technical ReportMiniMax Sparse AttentionDeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language ModelsConditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language ModelsKimi K2: Open Agentic IntelligenceQwen3 Technical ReportDeepSeek V3 Technical ReportDeepSeek V2: A Strong, Economical, and Efficient Mixture of Experts Language ModelDeepSeek V4: Towards Highly Efficient Million Token Context IntelligenceGLM 5: from Vibe Coding to Agentic Engineering
MoE Systems
4MoE 训练或推理中的通信、负载均衡与部署。
MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoEMegaScale MoE: Large Scale Communication Efficient Training of Mixture of Experts Models in ProductionDeepSeek V2: A Strong, Economical, and Efficient Mixture of Experts Language ModelUltraEP: Unleash MoE Training and Inference on Rack Scale Nodes with Near Optimal Load Balancing
Multi-Token Prediction
5一次 backbone step 预测多个 token 或 draft。
EAGLE 3: Scaling up Inference Acceleration of Large Language Models via Training Time TestDSpark: Confidence Scheduled Speculative Decoding with Semi Autoregressive GenerationDFlash: Block Diffusion for Flash Speculative DecodingDeepSeek V3 Technical ReportBreaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling
Memory Architecture
3模型内部的检索记忆、压缩状态或动态记忆结构。
Multimodal Model
2联合文本、图像或视频输入的模型与训练。
5 routes · 27 notes
Agents
Agent 程序、工具、记忆、协作与软件工程。
Agent Workflow
13表达和执行多阶段 agent 数据流与生命周期。
TacoMAS: Test Time Co Evolution of Topology and Capability in LLM based Multi Agent SystemsTowards Long Horizon Agents: A SurveySelf Compacting Language Model AgentsEfficient Serving for Agentic LLM Workflows via Micro Task Level ParallelismLLM as a Verifier: A General Purpose Verification FrameworkSPORK: Self Speculative Forking to Accelerate Agentic LLM InferenceComputer Environments Elicit General Agentic Intelligence in LLMsThunderAgent: A Simple, Fast and Program Aware Agentic Inference SystemSPIRAL: Learning to Search and AggregateECHO: Prune to act, trace to learn with selective turn memory in agentic RLSelf Improving Agents in the Era of Experience: A Survey of Self to Meta EvolutionUsing Span Queries to Optimize for Cache and Attention LocalityParrot: Efficient Serving of LLM based Applications with Semantic Variable
Tool Use
8模型调用 API、搜索、代码环境或计算机工具。
Multi Turn On Policy Distillation with Prefix ReplaySPORK: Self Speculative Forking to Accelerate Agentic LLM InferenceQwen3 Coder Next Technical ReportComputer Environments Elicit General Agentic Intelligence in LLMsThunderAgent: A Simple, Fast and Program Aware Agentic Inference SystemKimi K2: Open Agentic IntelligenceOn Effectiveness and Efficiency of Agentic Tool calling and RL TrainingKimi K2.5: Visual Agentic Intelligence
Agent Memory
8Agent 对历史 turn、经验与上下文的选择和重建。
RubricEM: Meta RL with Rubric guided Policy Decomposition beyond Verifiable RewardsTowards Long Horizon Agents: A SurveySmoothAgent: Efficient Long Horizon LLM Based Agent Serving with Lookahead Context EngineeringSelf Compacting Language Model AgentsCompactionRL: Reinforcement Learning with Context Compaction for Long Horizon AgentsECHO: Prune to act, trace to learn with selective turn memory in agentic RLSelf Improving Agents in the Era of Experience: A Survey of Self to Meta EvolutionDeepSeek V3.2: Pushing the Frontier of Open Large Language Models
Multi-Agent Orchestration
2多个 agent 或 subagent 的并行、聚合与编排。
Coding Agent
6面向 repository、测试和软件工程任务的 agent。
4 routes · 15 notes
Evaluation
验证器、过程监督、基准与能力边界分析。
Verifier
5对答案、轨迹、行为或候选进行评分与选择。
LLM as a Verifier: A General Purpose Verification FrameworkMath Shepherd: Verify and Reinforce LLMs Step by step without Human AnnotationsLet's Verify Step by StepSelf Trained Verification for Training and Test Time Self ImprovementMonitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
Process Supervision
3对中间 reasoning step 提供标签、reward 或验证。
Benchmark
7构造或审计用于比较模型和 agent 的评测集合。
Computer Environments Elicit General Agentic Intelligence in LLMsIf LLMs Have Human Like Attributes, Then So Does Age of Empires IIOn Effectiveness and Efficiency of Agentic Tool calling and RL TrainingImpossibleBench: Measuring LLMs' Propensity of Exploiting Test CasesFrom $f(x)$ and $g(x)$ to $f(g(x))$: LLMs Learn New Skills in RL by Composing Old OnesDoes Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?Spurious Rewards: Rethinking Training Signals in RLVR
Reasoning Analysis
5区分采样、组合、泛化与真实 reasoning 能力变化。
HydraHead: From Head Level Functional Heterogeneity to Specialized Attention HybridizationIf LLMs Have Human Like Attributes, Then So Does Age of Empires IIOn the Interplay of Pre Training, Mid Training, and RL on Reasoning Language ModelsFrom $f(x)$ and $g(x)$ to $f(g(x))$: LLMs Learn New Skills in RL by Composing Old OnesDoes Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
2 routes · 11 notes
Safety
奖励攻击、对齐、监控与高风险行为。
Reward Hacking
8优化 proxy、测试或规则时偏离真实目标。
Large Language Models Hack Rewards, and SocietyFrom Curiosity to Caution: Mitigating Reward Hacking for Best of $N$ with PessimismImpossibleBench: Measuring LLMs' Propensity of Exploiting Test CasesInference Time Reward Hacking in Large Language ModelsSpurious Rewards: Rethinking Training Signals in RLVRMonitoring Reasoning Models for Misbehavior and the Risks of Promoting ObfuscationCorrelated Proxies: A New Definition and Improved Mitigation for Reward HackingBeyond Reward Hacking: Causal Rewards for Large Language Model Alignment
AI Safety
10模型或 agent 的对齐、风险、欺骗与治理。
Towards Long Horizon Agents: A SurveySelf Improving Agents in the Era of Experience: A Survey of Self to Meta EvolutionReinforcement Learning Towards Broadly and Persistently Beneficial ModelsLarge Language Models Hack Rewards, and SocietyFrom Curiosity to Caution: Mitigating Reward Hacking for Best of $N$ with PessimismImpossibleBench: Measuring LLMs' Propensity of Exploiting Test CasesInference Time Reward Hacking in Large Language ModelsMonitoring Reasoning Models for Misbehavior and the Risks of Promoting ObfuscationCorrelated Proxies: A New Definition and Improved Mitigation for Reward HackingBeyond Reward Hacking: Causal Rewards for Large Language Model Alignment
3 routes · 16 notes
Theory
学习目标、复杂性与缩放规律的形式化结果。
RL Theory
13RL objective、估计量、方差与性能界。
Beyond Trajectory Level Attribution: Graph Based Credit Assignment for Agentic Reinforcement LearningDRPO: Efficient Reasoning via Decoupled Reward Policy OptimizationMaximum Likelihood Reinforcement LearningFrom Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language ModelsDo We Need to Verify Step by Step? Rethinking Process Supervision from a Theoretical PerspectiveThe Optimal Token Baseline: Variance Reduction for Long Horizon LLM RLVIMPO: Value Implicit Policy Optimization for LLMsTrust Region Masking for Long Horizon LLM Reinforcement LearningThe Entropy Mechanism of Reinforcement Learning for Reasoning Language ModelsApproximating KL DivergenceBroRL: Scaling Reinforcement Learning via Broadened ExplorationInference Time Reward Hacking in Large Language ModelsCorrelated Proxies: A New Definition and Improved Mitigation for Reward Hacking
Formal Expressivity
1模型相对逻辑、自动机或其他计算模型的表达复杂度。
Scaling Laws
2参数、数据、计算和 loss 之间的经验或理论缩放关系。