Should AI Research Try to Model the Human Brain?

TL;DR

Deep reinforcement learning can be made more efficient by incorporating episodic memory and meta reinforcement learning, concepts that are also found in the human brain.

Transcript

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. AI research has come a long-long way in the last few years. I remember that not so long ago, we were lucky if we could train a neural network to understand traffic signs, and since then, so many things happened: by harnessing the power of learning algorithms, we are now able... Read More

Key Insights

• 👨‍🔬 AI research has progressed significantly, but debates exist about the most promising research direction.
• 🐢 Deep reinforcement learning is powerful but slow due to incremental parameter adjustment and weak inductive bias.
• 🤘 Incorporating episodic memory and meta reinforcement learning can improve the efficiency of deep reinforcement learning.
• 🤘 These changes also align with mechanisms found in the human brain, suggesting a fundamental principle of meta reinforcement learning.
• 🧠 Understanding the human brain can provide insights and inspiration for more efficient AI algorithms.
• 🍉 Deep reinforcement learning should be evaluated in terms of efficiency and its alignment with human-like learning processes.
• 🏑 The proposed changes in deep reinforcement learning offer potential solutions and advancements in the field.

Questions & Answers

Q: What are the two changes proposed to improve the efficiency of deep reinforcement learning?

The two changes are incorporating episodic memory to store previous experiences and estimate the value of different actions, and letting the AI implement its own reinforcement learning algorithm ("learning to learn").

Q: How does using episodic memory improve the efficiency of deep reinforcement learning?

Episodic memory allows for drastic parameter adjustments based on previous experiences, enabling faster learning and decision-making. Studies have shown that using episodic memory also contributes to the learning of humans and animals.

Q: What is meta reinforcement learning?

Meta reinforcement learning refers to the AI implementing its own reinforcement learning algorithm. This helps the agent acquire more general knowledge that can be applied across tasks, further improving efficiency.

Q: How does deep reinforcement learning relate to the human brain?

The proposed changes in deep reinforcement learning align with similar mechanisms found in the human brain, specifically in neural structures involved in meta reinforcement learning. This suggests that meta reinforcement learning may be a fundamental principle of the human brain.

Summary & Key Takeaways

• AI research has made significant progress in recent years, but there is a debate about the most promising research direction.

• Deep reinforcement learning, while powerful, is often slow due to incremental parameter adjustment and weak inductive bias.

• This paper suggests that incorporating episodic memory and meta reinforcement learning can improve the efficiency of deep reinforcement learning and align it with the principles of the human brain.