To properly test whether language models contextualize word meanings, you need controlled experiments with words that stay the same but change meaning—this toolkit provides the infrastructure to do that rigorously.
This paper documents a toolkit for measuring how transformer language models handle words with multiple meanings across different contexts. The toolkit uses 'bridge forms'—words that appear unchanged but with different meanings in different domains—to test whether models truly distinguish word senses in their internal layers.