Infinite Context Length with Global but Constant Attention Memory
Junsong Chen, Yuyang Zhao, Jincheng Yu, Enze Xie
November 24, 2025
TL;DR
By reducing complexity from O(N^2) to O(N), Linear Attention is the key to processing ultra-long sequences. This post explores its mathematical core—"state accumulation"—and how it unlocks infinite context for LLMs and long video generation.
