Self-attention alone is sufficient for sequence modeling — the Transformer architecture became the foundation for virtually all modern language models.
Introduces the Transformer architecture, replacing recurrence and convolutions entirely with self-attention mechanisms. The model achieves state-of-the-art translation quality while being significantly more parallelizable and faster to train.