George Hotz: GPT writes sh*tty code | Lex Fridman Podcast Clips

TL;DR

Language models are rapidly developing and becoming human-like, potentially leading to transformative effects on society.

Transcript

so what do you think about this the rapid development of LMS if you could just like stick on that it's still incredibly impressive like with Chad GPT just even tragedy what are your thoughts about uh reinforcement learning with human feedback on these large language models to go back to when calculators first came out and or computers and like I wa... Read More

Key Insights

• 🇹🇩 The rapid development of language models, like Chad GPT, and their reinforcement learning with human feedback showcases impressive advancements.
• 🍝 Similar to past transformative technologies, such as calculators and computers, language models have the potential to dramatically impact society.
• 👨‍💻 Language models like Codex can assist in generating code, although the quality may be mediocre, providing benefits in terms of speed and efficiency.
• 👻 Future language models may incorporate retrieval systems with memory capabilities, allowing them to cite sources and integrate human supervision.

Questions & Answers

Q: How do language models compare to previous transformative technologies like calculators and computers?

Language models, like calculators and computers, have the potential for transformative effects on society. While calculators improved efficiency and computers revolutionized various industries, language models have the capability to transform human-like interactions and generate code, both with advantages and limitations.

Q: Are language models generating code of high quality?

Language models like Codex can generate code, but the quality may be mediocre. Despite this limitation, their ability to assist with writing boilerplate code and speeding up programming tasks can be valuable. However, caution should be exercised when relying on language models for critical code development.

Q: How can language models use retrieval systems to improve their performance?

Future language models are likely to incorporate retrieval systems with memory capabilities. These systems can allow language models to cite explicit sources, enhancing human supervision and preventing hallucinations. This integration of human knowledge and machine learning can lead to more reliable and trustworthy language models.

Q: What is the potential impact of language models that can cite sources?

Language models that can cite their sources and offer retrieval capabilities have the potential to challenge search engines like Google. By allowing human supervision and verification of information, these models can provide more reliable and accurate results, which may impact traditional search engines.

Summary & Key Takeaways

• The rapid development of language models (LMS) is impressive, particularly with models like Chad GPT and reinforcement learning with human feedback.

• Similar to the impact of calculators and computers in the past, the internet and language models show potential for transformative effects on society.

• Language models like Codex can generate code, although it may be mediocre, it still holds value in assisting with programming tasks.