Tian Liang | Chlience Paper Archive

Tian Liang

Equal-contribution author of FlashMemory-DeepSeek-V4 with paper-time affiliation Tencent. Author of HiLS-Attention with paper-time affiliation Tencent HY Team. Personal homepage identifies Tian Liang as a researcher at Tencent AI Lab and records prior education at Wuhan University and Tsinghua SIGS; high-confidence X was previously verified through homepage/profile consistency.

2 archived notes X: high Homepage GitHub X

Jul 09, 2026

Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling

HiLS Attention 把 chunk wise sparse attention 的选择问题改写为可端到端训练的 hierarchical softmax：每个 chunk 用 landmark token 生成压缩 key 和 entropy bias，query 先给 chunk 分配总 attention mass，再在被选 chunk 内做 token level attention。它在 345M 到 7B 实验中同...

2607.02980-hils-attention-infinite-context Sparse Attention Long Context

Jul 05, 2026

FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention

FlashMemory DeepSeek V4 在 DeepSeek V4 Flash 的 HCA / CSA 结构上增加一个 lookahead Neural Memory Indexer，把历史 CSA chunk 放入 CPU cold pool，再按未来约 $64$ 个 token 的预测需求取回少量 chunk；它把平均物理 KV 占用降到 full context baseline 的 $13.5\%$，平均分数从 $76...

2606.09079-flashmemory-deepseek-v4-lookahead-sparse-attention KV Cache Sparse Attention Long Context

Related Notes

Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling

FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention