tracked author

Samyam Rajbhandari

Recurring DeepSpeed author in the local archive: ZeRO coauthor and DeepSpeed Ulysses coauthor. Public sources identify him as DeepSpeed co-founder / system architect at Microsoft and later AI Systems Lead at Snowflake; X bio aligns with Snowflake and MSFT DeepSpeed.

2 archived notes X: high X

Related Notes

按论文归档时间排序，展示该作者在本站已经出现的材料。

归档 Jun 21, 2026

DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models

用 sequence sharded 与 head sharded 之间的 all to all 布局转换扩展极长序列训练。

待审阅 2309.14509-deepspeed-ulysses-long-sequence-training Distributed Training Long Context Attention Kernel

归档 Jun 18, 2026

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

按 data parallel rank 分片 optimizer state、gradient 与 parameter，降低大模型训练的显存复制。

待审阅 1910.02054-zero-memory-optimizations-trillion-parameter-models Training Memory Distributed Training