The idea that quantitative changes in a system (e.g., scaling up language models) can result in new behavior is known as emergence, a concept popularized by the 1972 essay “More is Different” by Nobel laureate Philip Anderson. Emergence has been observed in complex systems across many disciplines such as physics, biology, economics, and computer science.

An ability is emergent if it is not present in smaller models but is present in larger models.

For many tasks, model behavior either predictably grows with scale or unpredictably surge from random performance to above random at a specific scale threshold.

Emergent abilities are of scientific interest and motivate future research on large language models.