Linear-time sequence modeling is achievable without sacrificing quality — Mamba challenged the assumption that attention is necessary for strong language models.
Introduces Mamba, a selective state space model that achieves transformer-quality language modeling with linear-time complexity in sequence length. Uses input-dependent selection to filter information, overcoming the fundamental limitation of prior SSMs.