Scale produces emergent abilities — PaLM showed that certain capabilities like multi-step reasoning appear suddenly at sufficient scale rather than improving gradually.
Presents PaLM, a 540B parameter dense transformer trained on Google Pathways infrastructure across 6144 TPUs. Demonstrates breakthrough performance on reasoning tasks and shows that scaling continues to yield discontinuous capability improvements.