Nick Cheney: Capturing Neural Plasticity in Deep Networks

TL;DR

Researchers investigate how deep networks can adapt to changing environments and stimuli, resembling neuroplasticity in the brain.

Transcript

NICK CHENEY: I'm Nick Cheney. I'm finishing my PhD in Computational Biology at Cornell University. Gabriel Kreiman and I are interested in seeing how deep networks respond to neuroplasticity. So in the brain, we know that the brain is constantly in flux. And neurons are growing and dying. Weights are changing in response to stimuli. But most of the... Read More

Key Insights

• 🧠 Neuroplasticity in the brain involves constant changes, such as neuron growth and weight modifications in response to stimuli.
• 😑 Machine learning typically involves pre-training a network on a static dataset, but there is growing interest in online learning where the network adapts as it works with data.
• 👻 Researchers aim to identify rules that allow deep networks to maintain classification performance despite changes, similar to the brain's ability to adapt to everyday stimuli.
• ❤️‍🔥 Hebb's rule, where firing neurons strengthen connections, could potentially offer stable perturbations for deep networks.
• 🧠 Understanding the similarities and differences between deep networks and the brain is valuable for both scientific and engineering purposes.
• 👨‍🔬 This research has practical applications in fields like computer vision and neuroscience.
• 📦 Deep learning is currently a popular topic, and gaining experience with deep learning models and software packages is beneficial.

Questions & Answers

Q: What is the main focus of the research conducted by Nick Cheney and Gabriel Kreiman?

The main focus of their research is to understand how deep networks respond to neuroplasticity, specifically in the context of online learning.

Q: How are the researchers testing the network's ability to adapt to changes?

They are introducing random changes to the network's weights and analyzing how these perturbations affect its classification performance.

Q: What is Hebb's rule, and how is it related to the research?

Hebb's rule is a learning rule in neuroscience where neurons that fire consecutively strengthen their connections. The researchers aim to explore if similar rules can provide stable perturbations for the network to easily recover and maintain its performance.

Q: Why is understanding the similarity or difference between deep networks and the brain important?

It is scientifically intriguing to understand how computer models compare to the brain's functioning. Additionally, in the engineering context, as online learning becomes more prevalent, studying the stability of deep networks is crucial for future machine learning applications.

Summary & Key Takeaways

• Researchers at Cornell University are studying how deep networks respond to neuroplasticity, the brain's ability to change and adapt.

• They are conducting experiments by introducing random changes to the network's weights and observing how it affects image classification.

• The goal is to develop rules that allow the network to maintain its classification ability despite changes, similar to how the brain adapts to stimuli.