What Causes Hallucinations in Large Language Models?

TL;DR

Large language models (LLMs) can produce hallucinations because the likelihood of generating nonsensical responses increases exponentially with each token they produce. While fine-tuning helps LLMs perform better on common queries, the vast array of possible prompts means they remain prone to errors when encountering unfamiliar inputs.

Transcript

I think in one of your slides you have this nice plot that is one of the ways you show that llms are limited I wonder if you could talk about hallucinations from your perspectives the why hallucinations happen from large language models and why and to what degree is that a fundamental flaw of large language models right so because of the auto regre... Read More

Key Insights

• 🥺 The auto-regressive nature of large language models introduces the possibility of errors during token prediction, leading to hallucinations and nonsensical answers.
• 👾 Fine-tuning LLMs helps to cover common questions, but the immense space of possible prompts means there will always be scenarios that break the system.
• 🌥️ People have found that even minor changes or substitutions in prompts can cause large language models to produce incorrect answers.
• 💄 The curse of dimensionality makes it challenging for LLMs to naturally gravitate towards truth or reason.
• ❤️‍🩹 LLMs end up resembling lookup tables when they cannot reason or handle the long tail of unconventional prompts.
• ❓ Hallucinations and errors in LLM responses are a result of the probabilistic nature of token prediction.
• 😫 The exponential decrease in the probability of staying within the set of correct answers with each token produced contributes to the likelihood of nonsensical outputs.

Summary & Key Takeaways

• Large language models can produce erroneous answers as the probability of making mistakes accumulates exponentially with each token generated.

• Fine-tuning LLMs to cover a wide range of questions helps, but the space of possible prompts is enormous, and the system may break when faced with prompts it hasn't been trained on.

• Some prompts, even if reasonably grammatical, can break the system, such as substituting words with equivalents in another language.