How Can Collective AI Reduce Model Dependence?

TL;DR

Combining specialized AI models can produce a more capable and resilient system than depending on one large model or provider. Sakana AI pursues this collective-intelligence approach in resource-constrained Japan, while also researching systems that improve AI development itself and emphasizing human fact-checking as AI begins contributing to AI research.

Transcript

101 AI start. We're joined by David Hely of of Sakai. Thank you very much for the taking the time. >> Right. Thanks for having me here. It's truly an honor. >> Mhm. It's great to have you here. First uh I want to ask you uh about your journey leading up to founding the company and you started a career at Goldman Sachs and then uh became a AI resear... Read More

Key Insights

• David Ha's career connects finance and AI research: he studied neural networks at the University of Toronto, worked as a derivatives trader at Goldman Sachs in Japan, and later joined Google Brain after his independently published research attracted attention.
• World models allow agents to learn policies inside internally generated simulations or "hallucinations." At Google Brain, Ha collaborated with Jürgen Schmidhuber to modernize this concept and apply it to videos and video games as part of his generative AI research.
• Sakana AI was founded to give Japan a meaningful role in global AI development. Ha believed it would be regrettable for progress to be shaped primarily by the United States and China without input from Japanese culture, values, and technical communities.
• The name Sakana represents collective intelligence: a school of fish can display intelligence beyond any individual fish. The company applies that principle to AI by coordinating multiple models with different purposes, strengths, and weaknesses into one working system.
• Collective AI can reduce structural dependence on one model, vendor, company, or country. If a particular service becomes unavailable, a system built from multiple interchangeable models may continue operating by drawing on other components rather than stopping completely.
• Large monolithic models require substantial computing resources, energy, and data. Sakana AI pursues an alternative path in which smaller, specialized, and potentially local models cooperate, while recognizing that large general models still have valuable uses.
• Resource constraints can stimulate continuous improvement and efficient innovation. Ha sees Japan's comparatively limited resources as a suitable environment for developing creative AI methods, drawing a parallel with Japanese companies that innovated under constraints during the 1970s and 1980s.
• AI-assisted AI research can accelerate the creation of models and algorithms, but human fact-checking remains important. The interview presents human verification as a necessary safeguard when AI systems begin participating in research and generating findings that may require validation.

Questions & Answers

Q: Why did David Ha found Sakana AI in Japan?

David Ha founded Sakana AI in Japan because he wanted the country to participate in the global conversation about AI development. Although he is Canadian rather than Japanese, he values living in Japan and believes its culture and values should influence AI. He also sees Japan's limited resources as an advantage for developing efficient, constraint-driven alternatives to resource-intensive systems.

Q: What does collective intelligence mean at Sakana AI?

Collective intelligence means that useful intelligence can emerge from cooperation among many participants rather than residing in one individual system. Sakana AI illustrates the idea with a school of fish. In technical terms, the company seeks to coordinate multiple AI models, each with distinct purposes, strengths, and weaknesses, so their combined system can outperform or complement its individual components.

Q: How can multiple AI models reduce dependence on one provider?

A system that coordinates several AI models does not need to tie every function to one model, vendor, company, or country. Different models can be selected for the tasks they handle best, and alternative components can remain available if one service stops. This structure is intended to improve resilience while preserving access to several sources of capability.

Q: Why does Sakana AI challenge the focus on huge AI models?

Sakana AI challenges exclusive reliance on huge models because those systems require extensive computing resources, energy, and data. Ha believes strong AI may also emerge through cooperation among smaller or specialized models. His position does not reject large models entirely. Instead, it argues that efficient models and collective systems provide another path with different performance, resilience, and dependency benefits.

Q: How did David Ha move from finance into AI research?

David Ha studied neural networks at the University of Toronto near an AI winter, but a weak economy made a research career difficult, so he entered finance and became a derivatives trader at Goldman Sachs in Japan. He later read papers and conducted research during lunch breaks and after work. Recognition of work published on his anonymous blog eventually helped him join Google Brain.

Q: What AI research did David Ha conduct at Google Brain?

At Google Brain, David Ha worked on generative and creative AI. His projects included world models that let agents learn policies within internally generated simulations, as well as language-model-like methods for creative domains such as sketching. His group also explored smaller creative models and evolutionary optimization, focusing on innovation and energy efficiency rather than only increasing model size.

Q: Why is Japan suited to resource-efficient AI development?

Ha argues that Japan is suited to resource-efficient AI development precisely because it has fewer resources than the United States or China. Constraints can force researchers and companies to improve continuously and find more efficient methods. He compares this opportunity with Japanese companies such as Sony, Hitachi, and Panasonic, which he says produced major innovations despite limited resources during the 1970s and 1980s.

Q: Why is human fact-checking important when AI researches AI?

Human fact-checking remains important because AI systems are beginning to participate in researching and developing other AI systems. Outputs produced by an AI researcher can still require validation rather than automatic acceptance. The interview frames human verification as an essential part of this process, helping distinguish credible findings from claims that an AI system generated without sufficient factual support.

Summary & Key Takeaways

• David Ha moved from neural-network studies at the University of Toronto into finance, became a derivatives trader at Goldman Sachs in Japan, and independently resumed AI research. Recognition of his anonymous blog work eventually led him to Google Brain, where he researched generative AI, world models, creative systems, and evolutionary optimization methods.

• Ha founded Sakana AI because he wanted Japan to participate in the global conversation about AI development. The company name reflects its central idea: intelligence can emerge from a school of fish, not merely one fish. Likewise, multiple complementary models can cooperate instead of relying on one universal, monolithic system.

• Sakana AI views limited resources as a reason to pursue efficient, creative systems rather than imitate the largest American or Chinese projects. Its collective approach aims to improve performance, reduce dependence on a single model, vendor, company, or country, preserve operation during service disruptions, and support AI-assisted research with human verification.