Language models aren't just statistical pattern-matchers—they can provide genuine scientific insights into how language works, but only if we move beyond current limitations and integrate LM research with traditional linguistics.
This paper argues that language models can meaningfully contribute to linguistic science, despite common misconceptions. The authors address two main criticisms: the false belief that statistical models can't be linguistically interesting, and the assumption that current LM research represents the full potential for understanding language.