"Why You Believe The Things You Do: Evaluating ChatGPT vs. Google on 500 Search Queries"

"Why You Believe The Things You Do: Evaluating ChatGPT vs. Google on 500 Search Queries"

In our quest to understand why we believe the things we do, we decided to evaluate two prominent search engines: ChatGPT and Google. Surprisingly, our findings revealed that ChatGPT outperformed Google on coding queries and reached a tie on general informational queries, despite not being optimized for a search experience. This discovery led us to delve deeper into each platform, uncovering intriguing insights.

When examining ChatGPT's performance, we noticed that it received higher ratings at the extremes compared to Google. Users preferred ChatGPT on 42% of the queries, while Google garnered 40% preference. The dominance of ChatGPT became even more evident when we focused on coding-specific queries, where it outshined Google 70% of the time. This highlights ChatGPT's ability to synthesize information from various sources, providing a coherent and personalized search experience.

On the positive side, ChatGPT offers a minimal interface, sparing users from scrolling through numerous ads before finding the desired information. Additionally, it excels at understanding complex queries, demonstrating its capability to differentiate between similar concepts and provide accurate results. For instance, ChatGPT can identify that the "largest active volcano in the mainland of the United States" specifically excludes volcanoes in Hawaii.

However, ChatGPT does have its drawbacks. It occasionally produces hallucinations and inaccuracies, presented with unwavering confidence. While images, videos, and tweets hold importance in certain scenarios, ChatGPT's reliance solely on text-based responses may limit its ability to provide comprehensive answers.

Our exploration into why we believe certain things uncovered fascinating psychological factors. We discovered that our beliefs are influenced by our desire for them to be true. The more something helps us deal with uncertainty, the lower the threshold for believing it. This phenomenon is especially evident during times of crisis, such as the Great Plague of London, where people turned to prophecies and superstitions for hope.

Furthermore, we encountered Gibson's Law, which humorously states that for every PhD argument, there is an equal and opposite PhD argument. This highlights the availability of experts willing to support any belief or claim, often driven by personal motives rather than the pursuit of truth. Beliefs serve various purposes, such as justifying past actions, protecting reputations, providing hope, maximizing income, and signaling tribal belonging.