Paper Archive

输入关键词搜索 102 篇论文

tracked author

Size Zheng

ByteDance Seed systems/compiler researcher. Homepage, Scholar, OpenReview, and DBLP connect Size Zheng to MegaScale-MoE, TileLink, Triton-distributed, and distributed AI systems. No high-confidence personal X account found in this pass.

1 archived notes X: not-found Homepage

Related Notes

按论文归档时间排序，展示该作者在本站已经出现的材料。

Jun 28, 2026

MegaScale MoE: Large Scale Communication Efficient Training of Mixture of Experts Models in Production

MegaScale MoE 的核心贡献是把大规模 MoE 训练的瓶颈从单点 kernel 优化提升到整层通信路径设计：attention 侧用 Ulysses style sequence parallelism 降低 TP critical path 通信，FFN 侧用 intra node expert parallelism 保持专家 GEMM 效率，再用跨算子调度、算子内 tile level overlap、选择性 acti...

2505.11432-megascale-moe-communication-efficient-training MoE SystemsDistributed TrainingTraining Memory