George Hotz: I'd put money on Mark Zuckerberg | Lex Fridman Podcast Clips

TL;DR

Language models with memory systems and retrieval systems have the potential to revolutionize search engines, allowing for explicit citation of sources and human supervision. Startups may pose a threat to established giants like Google, with Mark Zuckerberg and Elon Musk being potential contenders.

Transcript

it's actually weird the way that we do uh language models right now are all of the uh information is in the weights and the human brains don't really like this with like a hippocampus and a memory system so why don't llms have a memory system and there's people working on them I think future LMS are going to be like smaller but are going to run loo... Read More

Key Insights

• 👨‍🔬 Future LLMs with memory and retrieval systems have the potential to revolutionize search engines by enhancing human involvement and enabling explicit source citation.
• 👨‍🔬 Startups could disrupt the search engine market by developing compelling products and leveraging user data to train future iterations of their models.
• 🙈 Mark Zuckerberg and Facebook's Meta are seen as potential winners in the search engine race due to their ability to innovate and pivot aggressively.
• ✊ The capitalist structure and power dynamics make it difficult to have a button to turn off established companies like Google, but startups may have more control over their platforms.
• 🧑‍🏭 The dominance of a search engine relies on factors like user behavior, data, and the ability to provide accurate and relevant results.
• 💦 Establishing a search engine that works nicely and stably with explicit source citation can be challenging but potentially lucrative.
• 🚄 The winner-takes-all effect plays a role in the search engine market, as the company with high usage will have access to the data needed to train future models.

Questions & Answers

Q: How can future LLMs improve search engines?

Future LLMs can incorporate memory systems and retrieval systems, allowing for explicit source citation. This would enhance the human involvement in the search process and provide a mechanism for oversight and correction of hallucinations or inaccuracies.

Q: Can LLMs capable of citing sources replace Google?

Yes, if an LLM is developed with the ability to cite sources, it can essentially function as a search engine. This could pose a threat to Google's dominance, as people desire a search engine that explicitly cites reliable sources.

Q: Who are the competitors in the search engine market?

Apart from established players like Google, startups like tiny Corp can pose competition. However, innovative startups with compelling products and features could disrupt the industry in the future.

Q: Does Google have the advantage due to its size and resources?

While Google's size and resources provide advantages, the author bets on individuals like Mark Zuckerberg and startups due to their ability to pivot and innovate more aggressively. The author believes that the startup approach is more likely to create a search engine that can compete with or surpass Google.

Summary & Key Takeaways

• Language models (LLMs) currently lack a memory system, unlike human brains, but future LLMs may have looping capabilities and retrieval systems, enabling explicit source citation and human supervision.

• The ability of LLMs to cite sources like a search engine could potentially replace Google, but Google might also be the one to develop this capability and monetize it.

• The competition in the search engine market includes contenders like tiny Corp, but startups with innovative approaches may disrupt the industry.

• The author believes that Google's dominance in search may decline by the end of the decade, with Mark Zuckerberg and Facebook's Meta being potential winners.

• The question of whether the authority exists to shut down companies like Google or Facebook reveals the power dynamics within capitalist structures.