Test-time regression offers a more effective way to compress token sequences than standard pooling or projection methods, enabling better efficiency in both language and multimodal models without sacrificing performance.
This paper proposes TTR-Compressor, a new method for compressing sequences of tokens in language and multimodal models. Instead of using pooling or learned projections, it frames compression as a regression problem solved at test time with coefficients computed efficiently in closed form.