Animals — including humans — don’t use labeled data sets to learn. For the most part, they explore the environment on their own, and in doing so, they gain a rich and robust understanding of the world.

“self-supervised learning” algorithms have proved enormously successful at modeling human language and, more recently, image recognition.

Self-supervised algorithms essentially create gaps in the data and ask the neural network to fill in the blanks.

In a so-called large language model, for instance, the training algorithm will show the neural network the first few words of a sentence and ask it to predict the next word. When trained with a massive corpus of text gleaned from the internet, the model appears to learn the syntactic structure of the language, demonstrating impressive linguistic ability — all without external labels or supervision.

Biological brains are thought to be continually predicting, say, an object’s future location as it moves, or the next word in a sentence, just as a self-supervised learning algorithm attempts to predict the gap in an image or a segment of text.

our visual system has two specialized pathways because they help predict the visual future, said Richards; a single pathway isn’t good enough.

Truly understanding brain function is going to require more than self-supervised learning. For one thing, the brain is full of feedback connections, while current models have few such connections, if any.