Language models can learn the semantic meaning of rare grammatical constructions, but this semantic understanding emerges later in training than syntactic knowledge and correlates with broader world knowledge gains.
This paper tests whether language models understand rare English constructions like "let alone" and "much less"—not just their grammar, but their actual meaning.