Xu Shen | Chlience Paper Archive

Xu Shen

HydraHead coauthor. arXiv lists Alibaba Group. Google Scholar identifies Tongyi Large Model Business Unit, Alibaba Token Hub with verified alibaba-inc.com email. Public OpenReview snippet identifies Alibaba Group researcher history and USTC PhD student history.

1 archived notes X: not-found Homepage

Jul 02, 2026

HydraHead: From Head Level Functional Heterogeneity to Specialized Attention Hybridization

HydraHead 把 Full Attention (FA) 和 Linear Attention (LA) 的混合粒度从 layer 推进到 attention head：先用 activation patching 和 path patching 找到 retrieval critical heads，只给这些头保留 FA，其余头换成 Gated DeltaNet (GDN)，再用独立 RMSNorm 与 head wise s...

2606.20097-hydrahead-head-wise-hybrid-attention Linear Attention Long Context Reasoning Analysis

Related Notes

HydraHead: From Head Level Functional Heterogeneity to Specialized Attention Hybridization