Lei Zhu (祝磊)

Lei Zhu (祝磊)

Author of HiLS-Attention with paper-time affiliation Tencent HY Team and author of MoESD with paper-time affiliation Huawei Noah’s Ark Lab. Homepage identifies ZHU Lei (祝磊) as a senior researcher at Tencent Hunyuan Frontier Lab focused on long-term memory, long-context modeling, sparse attention and LLM inference acceleration; its selected-publication list highlights MoESD, RelayAttention and BiFormer. No high-confidence personal X account found.

2 archived notes X: not-found Homepage GitHub

归档 Jul 24, 2026

MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE

在单步解码已使专家加载趋于饱和、模型仍受内存带宽限制的中等批量区间，用投机验证复用已加载专家；Qwen2 57B A14B 在两张 H800 上最高加速 2.29 倍，适用范围依赖路由均衡、MoE 前馈网络成本占比和具体硬件。

待审阅 2505.19645-moesd-sparse-moe-speculative-decoding Speculative Decoding MoE Systems Serving Runtime

归档 Jul 09, 2026

Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling

HiLS Attention 用 landmark 压缩键、熵偏置与分层 softmax 训练 chunk 选择器，使 8K 训练的 345M 模型在 4M RULER 单针检索仍得 96 分，并在 512K 单 H800、batch size 1 的同 Triton 基线上将 prefill 与 decode 分别加速 13.5 倍和 15.7 倍。

待审阅 2607.02980-hils-attention-infinite-context Sparse Attention Long Context

Representative Papers

Related Notes

MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE

Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling