Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)

TL;DR

This talk discusses the need for machine learning systems to handle changes in distribution, prioritize high-level cognition, and incorporate attentional mechanisms for improved performance.

Transcript

thank you so much for inviting me for this talk and thank you so much for being so numerous today I have a lot of things to talk about but what's interesting in my mind is that these things are linked together in a really interesting way at least that's what I'll try to convince you about so the title talks about system to cognition which is about ... Read More

Key Insights

• 🍵 Deep learning needs to improve out-of-distribution generalization and adaptability to handle changes in data distribution.
• 🤩 Attention mechanisms are key to enabling high-level cognition and improving deep learning performance.
• 🤔 Consciousness can be thought of as a bottleneck of information, where selected elements are broadcasted and influence the rest of the brain.
• ✋ The structure of high-level semantic variables can be represented by a sparse factor graph, aiding in better understanding and modeling of the world.
• 👻 Changes in distribution can be localized and explained through interventions by an agent, allowing for efficient adaptation and learning.
• 😫 Neural nets that operate on sets of objects and incorporate dynamically recombined modules can enhance compositionality and reasoning abilities.

Questions & Answers

Q: What is the main challenge with current machine learning systems?

Current machines learn narrowly, require large amounts of data to learn new tasks, and are not robust to changes in distribution, resulting in mistakes and inefficiency.

Q: How can deep learning be extended to approach human-level AI?

By incorporating attention mechanisms and improving generalization abilities, deep learning can tackle high-level cognition, discover high-level representations, and understand cause and effect.

Q: What advantages does human-level intelligence have over machines?

Humans can generalize efficiently, handle high-level cognitive tasks, and understand causality thanks to their ability to combine systems one and two in learning and reasoning.

Q: How can machine learning help us understand consciousness?

Machine learning can provide mechanistic explanations and formalize cognitive functionalities, guiding neuroscience research and expanding our knowledge of consciousness.

Summary & Key Takeaways

• Deep learning has made significant progress, but there is a need for qualitative advancements to approach human-level AI.

• Current machines learn narrowly and require large amounts of data, while still making mistakes and struggling with changes in distribution.

• The speaker proposes a path to extend deep learning abilities to tackle high-level cognition through attention mechanisms, better generalization, and modeling causal relationships.