What we learned from 5 million books - Erez Lieberman Aiden and Jean-Baptiste Michel

TL;DR

By analyzing data from millions of digitized books, researchers have discovered insights into human culture and history, demonstrating the power of digitization.

Transcript

[Applause] everyone knows that a picture is worth a thousand words but we at Harvard were wondering if this was really true so we assembled a team of experts spanning Harvard MIT the American Heritage dictionary the encyclopedia britanica and even our proud sponsors the Google and we cogitated about this for about four years and we came to a startl... Read More

Key Insights

• 📔 Digitization of books has made it possible to analyze and extract insights from millions of books, transforming the way we understand human culture and history.
• 👻 Analysis of the two billion engrams allows for the measurement of cultural trends and changes over time.
• 📔 Digitized books provide a valuable resource for understanding language usage and the evolution of cultural preferences.
• 🕵️ Censorship and suppression can be detected through analysis of fame and popularity trends in the historical record.
• 🏑 Historical figures in different fields, such as politics, literature, and science, experience varying trajectories of fame and recognition.
• 📔 Cultural trends and phenomena, such as influenza epidemics and climate change, can be observed and analyzed through the data from digitized books.
• 🍉 Different languages and cultures may exhibit unique patterns and fluctuations in terms of fame and popularity.

Questions & Answers

Q: How did researchers determine that some pictures are worth 500 billion words?

By analyzing data from millions of digitized books, researchers were able to extract information and insights that would otherwise be inaccessible. This enabled them to determine the impact and significance of certain pictures in human culture and history.

Q: What kind of information can be derived from digitized books?

Digitized books provide a wealth of information, including metadata such as publication details, author information, and publication dates. Additionally, by analyzing the text itself, researchers can gain insights into cultural trends, language usage, and historical events.

Q: How did researchers handle the challenge of releasing the full text of millions of books?

Releasing the full text of millions of books posed practical and legal challenges, so researchers opted to release statistics about the books instead. By focusing on word frequencies and usage over time, they were still able to gain valuable insights into cultural change and trends.

Q: What is the significance of the two billion engrams?

The two billion engrams represent a rich dataset that captures the way culture has evolved over time. By analyzing the frequency of different words and phrases, researchers can gain insights into language usage, cultural preferences, and historical events.

Summary & Key Takeaways

• Researchers assembled a team of experts to analyze the impact of pictures in human culture and history, concluding that some pictures are worth 500 billion words.

• Google's digitization project has made it possible to use computational methods to analyze and extract information from millions of books, providing a practical and extensive resource.

• By releasing statistics about the books instead of the full text, researchers were able to create a database of two billion engrams, revealing cultural trends over time.