"He was painstakingly designing a system that could recognize objects in pictures, regardless of variations in size, position and other properties — something that humans do with ease."

"The system was a deep neural network, a type of computational device inspired by the neurological wiring of living brains."

"They have wondered not just why different parts of the brain do different things, but also why the differences can be so specific: Why, for example, does the brain have an area for recognizing objects in general but also for faces in particular?"

"The computational neuroscientist Daniel Yamins, now at Stanford University, showed that a neural network processing the features of a scene hierarchically, much as the brain does, could match the performance of humans at recognizing objects."

"Similarly, researchers have demonstrated that the deep networks most proficient at classifying speech, music and simulated scents have architectures that seem to parallel the brain’s auditory and olfactory systems."

"Sandwich one or more “hidden” layers between the input and the output and you get a “deep” neural network; the greater the number of hidden layers, the deeper the network."

"In their general structure and function, deep nets aspire loosely to emulate brains, in which the adjusted strengths of connections between neurons reflect learned associations."

"The key neuroscientific insight is that visual information processing is hierarchical and proceeds in stages: The earlier stages process low-level features in the visual field (such as edges, contours, colors and shapes), whereas complex representations, such as whole objects and faces, emerge only later in the inferior temporal cortex."

"His team, which included Alexander Kell and Yamins, began designing deep nets to classify two types of sounds: speech and music."

"First, they hard-coded a model of the cochlea — the sound-transducing organ in the inner ear, whose workings are understood in great detail — to process audio and sort the sounds into different frequency channels as inputs to a convolutional neural network."

"Three sets of architectures seemed possible. The deep net’s two tasks could share only the input layer and then split into two distinct networks. At the other extreme, the tasks could share the same network for all their processing and split only at the output stage."

"Why does the brain segregate the processing of faces from that of other objects?"

"The team found that the deep net trained to recognize faces was bad at recognizing objects and vice versa, suggesting that these networks represent faces and objects differently."

"They found that the network had internally organized itself to segregate the processing of faces and objects in the later stages of the network."

"“We found that functional specialization of face and object processing spontaneously emerged in deep nets trained on both tasks, like it does in the human brain,”"

"More such evidence is emerging from research tackling the perception of smells."

"The first layer of odor processing involves olfactory sensory neurons, each of which expresses only one of about 50 types of odor receptors."

"When Yang and colleagues trained this network to classify the simulated odors, they found that the network converged on much the same connectivity as seen in the fruit fly brain"

"This similarity suggests that both evolution and the deep net have reached an optimal solution."

"The next step in testing will be to evolve deep networks that can predict the connectivity in the olfactory system of some animal not yet studied, which can then be confirmed by neuroscientists."

"Isn’t a deep-net model of some part of the brain merely replacing one black box with another?"

"For instance, many of these models need loads of labeled data for training, while our brains can learn effortlessly from as little as one example."

"Deep nets also learn using an algorithm called back propagation, which most neuroscientists think cannot work in real neural tissue because it lacks the appropriate connections."

"Josh Tenenbaum, a cognitive neuroscientist at MIT, said that while all these deep-net models are “real steps of progress,” they are mainly doing classification or categorization tasks."

"Our brains, however, do much more than categorize what’s out there."

"This suggests that brains use combinations of generative and recognition models not just to recognize and characterize objects but to infer the causal structures inherent in scenes, all in an instant."

"Such “supervised” training requires data laboriously labeled by humans, and the neural networks often take shortcuts, learning to associate the labels with minimal and sometimes superficial information."

"Animals — including humans — don’t use labeled data sets to learn. For the most part, they explore the environment on their own, and in doing so, they gain a rich and robust understanding of the world."

"Now some computational neuroscientists have begun to explore neural networks that have been trained with little or no human-labeled data."

"These “self-supervised learning” algorithms have proved enormously successful at modeling human language and, more recently, image recognition."

"In recent work, computational models of the mammalian visual and auditory systems built using self-supervised learning models have shown a closer correspondence to brain function than their supervised-learning counterparts."

"Brain models inspired by artificial neural networks came of age about 10 years ago, around the same time that a neural network named AlexNet revolutionized the task of classifying unknown images."

"If a neural network fails to classify an image correctly, the learning algorithm updates the weights of the connections between the neurons to make that misclassification less likely in the next round of training"

"Around the same time, neuroscientists developed the first computational models of the primate visual system, using neural networks like AlexNet and its successors."

"Self-supervised algorithms essentially create gaps in the data and ask the neural network to fill in the blanks."

"The self-supervised learning algorithm trains the encoder-decoder combination to turn masked images into their full versions."

"Any differences between the real images and the reconstructed ones get fed back into the system to help it learn."

"“I think there’s no doubt that 90% of what the brain does is self-supervised learning,”"

"Richards’ team found that an AI trained with a single ResNet was good at object recognition, but not at categorizing movement."

"The activity in the early layers of the AI aligns with activity in the primary auditory cortex, whereas the activity of the AI’s deepest layers aligns with activity in the higher layers in the brain, in this case the prefrontal cortex."

"“It’s not conclusive, but [it’s] another compelling piece of evidence to suggest that, indeed, the way we learn language is in large part by trying to predict next things that will be said.”"

"Truly understanding brain function is going to require more than self-supervised learning. For one thing, the brain is full of feedback connections, while current models have few such connections, if any."

"An obvious next step would be to use self-supervised learning to train highly recurrent networks — a difficult process — and see how the activity in such networks compares to real brain activity."

"The other crucial step would be to match the activity of artificial neurons in self-supervised learning models to the activity of individual biological neurons."