How is Gemini 2.0 shaping agentic AI today?

TL;DR

Pretraining and imitation learning build a general digital brain, then reinforcement learning refines it using rewards. Gemini 2.0 aims for broader applications beyond single tasks by enabling models to act with user intent, learn from data, and improve through planning and reasoning as discussed by Oriol Vinyals.

Transcript

[MUSIC PLAYING] HANNAH FRY: Welcome to "Google DeepMind-- the Podcast." I'm Professor Hannah Fry. Right, agents, they are here, or almost here, and they are probably all anyone is going to be talking about in 2025. But they are definitely not new. My guest on today's episode is someone who last came on the podcast in 2019 to talk to me about the mu... Read More

Key Insights

• Gemini uses a two phase training process: pretraining or imitation learning followed by reinforcement learning post training.
• Agentic behavior emerges from combining multimodal inputs with planning and offline learning, enabling autonomous user aligned actions.
• The digital brain is iteratively refined while its weights remain largely frozen once deployed, emphasizing efficiency and stability.
• Architectural tweaks to transformers and other components continue to improve learning without changing core principles.
• Multimodality and online learning ideas enable models to gather information and refine strategies beyond static training data.
• Progression from narrow specialized agents to broad capability models relies on backporting future-thinking ideas into current practice.
• Rewards in reinforcement learning guide improvements without requiring direct human evaluation of every action.
• Memory, reasoning, and planning are identified as crucial components for future agentic capabilities.
• More videos from this channel:
  • How to predict value functions without a model
  • How to build AGI with a systems neuroscience approach
  • How to scale value functions with function approximators
• More from this channel: Google DeepMind

Questions & Answers

Q: What are the two main stages of training a modern agent like Gemini 2.0?

The two main stages are pretraining or imitation learning, where the model adapts its weights by mimicking data from humans and the internet, and post training or reinforcement learning, where the model further adjusts its weights based on reward signals to improve the quality of its outputs. The first stage builds broad capabilities, while the second sharpens behavior toward desirable goals.

Q: How does reinforcement learning differ from the initial imitation phase?

Reinforcement learning follows imitation learning and uses reward signals to adjust the model toward higher quality outputs or better strategies, rather than simply copying human data. This phase emphasizes discovering actions that yield positive results, such as better poems or more effective game moves, rather than matching existing examples alone.

Q: Why are weights described as the most precious part of the model after training?

Weights encode the learned behavior and knowledge of the model from the training process. Once training finishes, the weights define how the model responds to inputs, and they are kept frozen to ensure stability and efficiency. Modifying them later would constitute additional training, which is not desired for a deployed system.

Q: What role does multimodality play in Gemini 2.0 according to the interview?

Multimodality allows the model to process and integrate multiple input types such as text and images, enhancing understanding and interaction. This capability serves as the model feedback mechanism for user questions and tasks, enabling more natural and effective guidance and planning across different contexts.

Q: What is meant by the term global objective in the context of agentic AI?

The global objective refers to aligning the model's behavior with user intents and desired outcomes across diverse tasks. This involves using planning, reasoning, and memory to select actions that fulfill user goals while adhering to safety and reliability constraints, rather than optimizing for a single narrow metric.

Q: How might a model go online to improve its performance, as discussed in the episode?

A model could go online to learn by watching videos, reading forums, downloading games, and interacting with environments to gather information about its weaknesses. This online exploration would inform subsequent offline training, enabling the agent to refine strategies and then report updated capabilities to users.

Q: What is the significance of the two phase training approach for future AGI progress?

The two phase approach separates broad capability acquisition from goal aligned optimization. Pretraining builds general understanding from large data, while post training optimizes actions through rewards to achieve user aligned objectives. This separation supports scalable, stable improvements and allows for iterative advances without disrupting deployed systems.

Q: What challenges are associated with post training for language and multimodal agents?

Post training faces challenges in obtaining clean, reliable reward signals and aligning the agent with complex user intents. It also requires balancing exploration with safety, ensuring that the agent does not engage in undesired behaviors while still improving performance. These issues make reinforcement learning a careful, ongoing process.

Summary & Key Takeaways

• Gemini 2.0 builds a general digital brain through two main phases, imitation learning and reinforcement learning post training, to enable broader applications.

• The conversation explains that models are trained first on data, then refined by rewards, and finally deployed with frozen weights that users access.

• The episode highlights agentic capabilities and how architecture, scaling, and multimodal inputs contribute to more autonomous behavior.