In January, a group of young academics working in natural language processing — the branch of A.I. focused on linguistic understanding — issued a challenge to try to turn this paradigm on its head.

The group called for teams to create functional language models ‌using data sets that are less than one-ten-thousandth the size of those used by the most advanced large language models. A successful mini-model would be nearly as capable as the high-end models but much smaller, more accessible and ‌more compatible with humans. The project is called the BabyLM Challenge.

OpenAI’s GPT-3, released in 2020, was trained on 200 billion words; DeepMind’s Chinchilla, released in 2022, was trained on a trillion.

So, together with a half-dozen ‌colleagues, Dr. Wilcox, Dr. Mueller and Dr. Warstadt conceived of the BabyLM Challenge, to try to nudge language models slightly closer to human understanding. In January, they sent out a call for teams to train language models on the same number of words that a 13‌-year-old human ‌encounters — roughly 100 million. Candidate models would be ‌tested on how well they ‌generated and picked up the nuances of language, and a winner would be declared.

The potential of such a research program has not been ignored by bigger industry labs. Sam Altman, the chief executive of OpenAI, recently said that increasing the size of language models would not lead to the same kind of improvements seen over the past few years

companies like Google and Meta have also been investing in research into more efficient language models, informed by human cognitive structures. After all, a model that can generate language when trained on less data could potentially be scaled up, too.