The Future of AI: Twitter's Full Tweet Archive and the Generative AI Platform

The Future of AI: Twitter's Full Tweet Archive and the Generative AI Platform

In recent news, Twitter has announced that it will be opening up its full tweet archive to academic researchers for free. Previously, researchers had to pay for premium or enterprise developer access, but now, they will have access to the "full history of public conversation" through Twitter's full-archive search endpoint. This move comes as Twitter recognizes the importance of social media discourse in understanding online misinformation, election interference, hate speech, and other topics that have taken center stage in recent years.

Twitter's decision to provide free access to its tweet archive is a significant step in enabling researchers to delve deeper into the world of social media and gain insights that can help make the world a better place. For over a decade, academic researchers have used Twitter data for discoveries and innovations, and now, with easier access to the data they need, they can further contribute to our understanding of various phenomena happening on the platform.

The company is also increasing the monthly tweet volume cap for approved applicants to 10 million tweets, which is 20 times higher than the previous limit. This larger volume cap allows researchers to analyze a more extensive range of data and draw more comprehensive conclusions from their studies.

However, it's worth noting that Twitter will not be providing access to data from accounts that have been suspended or banned. While this decision may complicate efforts to study hate speech, misinformation, and other conversations that violate Twitter rules, it also underscores the platform's commitment to maintaining a safe and responsible environment for its users.

On a separate note, the generative AI platform has been experiencing remarkable growth, driven by its novelty and a wide range of use cases. Three product categories, namely image generation, copywriting, and code writing, have already exceeded $100 million in annualized revenue. This growth has led to the question of who owns the generative AI platform and how businesses can build sustainable models within this space.

Infrastructure vendors seem to be the biggest winners in the generative AI market, capturing the majority of the revenue flowing through the stack. Application companies, on the other hand, face challenges in retention, product differentiation, and gross margins. While they experience rapid revenue growth, they struggle to establish long-term customer value. Model providers, responsible for the existence of this market, have yet to achieve large commercial scale.

To build a sustainable generative AI business, it's essential to consider technical differentiation, network effects, data retention, and complex workflows. While selling end-user apps may seem like the most straightforward path, it's not necessarily the best one. Margins and retention can improve over time as competition increases and AI tourists exit the market. Additionally, vertically integrated apps have an advantage in driving differentiation.

One key takeaway for model providers is the recognition that commercialization is often tied to hosting services. Demand for proprietary APIs is growing rapidly, and hosting services for open-source models are emerging as useful hubs for sharing and integrating models. This highlights the importance of creating a robust hosting infrastructure to capture value within the generative AI market.

However, it's worth considering the ethical implications of generative AI. Many model providers have incorporated the public good explicitly into their missions, with some organizing as public benefit corporations. This commitment to the public good has not hindered their fundraising efforts, but it raises the question of whether capturing value should be the primary goal for model providers.

When it comes to revenue distribution in the generative AI market, a significant portion flows through to infrastructure companies. On average, app companies spend a substantial percentage of their revenue on inference and fine-tuning, either directly to cloud providers or third-party model providers who, in turn, spend a significant portion on cloud infrastructure. Nvidia, a leader in data center GPUs, is a notable winner in this space, running the majority of AI workloads behind the scenes.

Infrastructure companies benefit from standard moats such as scale, supply-chain, ecosystem, algorithmic, distribution, and data pipeline moats. However, the durability of these moats over the long term remains uncertain. It's also unclear whether a winner-take-all dynamic will emerge in the generative AI market. Both horizontal and vertical companies are expected to succeed, depending on the end-markets and end-users they cater to.

In conclusion, the future of AI is evolving rapidly, with Twitter's decision to open up its full tweet archive providing valuable opportunities for academic researchers. Simultaneously, the generative AI platform presents both challenges and opportunities for businesses. To navigate this landscape successfully, it's crucial to consider technical differentiation, hosting infrastructure, and the ethical implications of AI. By leveraging actionable advice, businesses can position themselves for success in the ever-changing world of AI and contribute to making the world a better place.

Actionable Advice:

1. Embrace open access to data: Researchers should take advantage of platforms like Twitter that provide free access to their archives. By utilizing this data, they can uncover valuable insights and contribute to societal advancements.
2. Establish robust hosting infrastructure: Model providers should focus on building a reliable and scalable hosting infrastructure. This will enable them to capture value within the generative AI market and meet the growing demand for proprietary APIs and hosting services.
3. Consider the public good: When developing AI models, businesses should incorporate the public good into their mission. By aligning their values with societal benefits, they can attract support and funding while making a positive impact on the world.