How Can AI Help Us Talk With Our Environment?

TL;DR

Large language models can turn complex data from companies, vehicles, species, cells, rivers, and planetary observation systems into conversational interfaces. By letting people ask follow-up questions and encounter unexpected insights, AI could give collective systems a comprehensible voice, deepen understanding, and connect people more closely with themselves and the world around them.

Transcript

[Applause] [Applause] [Applause] art asks questions that sound you just heard was the sound of the kyoga river do you remember that river that was the one that was so polluted it caught fire in Cleveland and that sound came from an art project that I had been working on with a group called spur back when I was in graduate school I was doing work wi... Read More

Key Insights

• Conversation is a learning process because it enables people to ask questions, receive responses, encounter surprises, and refine their understanding through dialogue. Its value goes beyond simple call and response because each exchange can reveal new information about oneself or the surrounding world.
• Large language models are user interfaces as well as computational tools. Earlier computers required users to learn specialized languages, while current models can interact through ordinary human language, making complex computation and stored information more approachable through conversation.
• A large language model is described as a probabilistic compression of cultural data represented on the web. This unusual structure allows people to hold dialogues with information rather than merely submit commands or receive isolated computational results.
• Company data can become conversational when a language model provides an interface for questioning stored information. A user can identify an unusual peak, ask why it occurred, connect it to a holiday, and continue investigating the meaning of that pattern.
• Collective intelligence can emerge from complex systems whose parts work together. White connects this idea to living cells, rivers, vehicles, animal communication, and environmental datasets, proposing that AI could translate their combined signals into forms people can question and understand.
• Giving a river a voice can change how people perceive it. White's graduate-school art project combined sampled river sounds, interviews collected along its curves and bends, and an immersive playback space to represent the Cuyahoga River as something resembling a collective intelligence.
• AI can function as a translator or transducer between humans and other species. White cites work with the Earth Species Project as an example of using language models to process a highly complex system and support attempts to communicate with whales.
• Planetary conversation would require combining multiple complex datasets rather than relying on a single source. White contrasts roughly seven petabytes used for Pi and many large language models with roughly 50 petabytes associated with Planet Labs, while noting that satellite observations represent only one dataset.

Questions & Answers

Q: How can AI help humans converse with the environment?

AI can organize signals from complex systems and present them through a conversational interface. People could ask questions, receive understandable responses, and pursue unexpected findings through follow-up dialogue. White proposes applying this pattern not only to web information or company data, but also to rivers, vehicles, living cells, whales, environmental observations, and potentially the planet as a whole.

Q: Why are large language models considered user interfaces?

Large language models let people interact with computation and data through ordinary language. White contrasts this with his experience learning 6502 assembly in 1981, when communicating with a computer required learning its specialized language. With language models, the computer instead learns human language, allowing users to investigate information through familiar questions and an extended conversational exchange.

Q: How can a language model make company data conversational?

A language model can provide a dialogue layer over information stored by a company. White illustrates this with a user asking what happened during the previous week, noticing an unusual peak, and then asking why it occurred. The system can connect the pattern to a holiday in Russia and support further questions about the result and its significance.

Q: What did Sean White's river art project demonstrate?

The project demonstrated how combining different forms of information can give a complex subject a perceived voice. White and the art collective SPUR sampled sounds from the Cuyahoga River, interviewed people along its curves and bends, mixed the material, and played it through an immersive space. Visitors could then experience the river as something resembling a collective intelligence.

Q: What does collective intelligence mean in the talk?

Collective intelligence refers to understanding a complex system as an integrated whole created by many interacting parts. White mentions research that treats cells and biological systems in this way, then extends the idea to rivers, cars, whales, and the planet. AI could combine their signals and make the resulting patterns accessible through a conversational interface.

Q: Can large language models help people communicate with whales?

White says language models are already being used with the Earth Species Project as part of efforts to communicate with whales. In this framing, AI acts as a translator or transducer for an extremely complex system. The goal is to process patterns from another species and create a form of exchange that humans can engage with conversationally.

Q: What would it take to have a conversation with the planet?

A conversation with the planet would require combining multiple large and complex environmental datasets, then using AI to turn their patterns into an interactive voice. White notes that the data associated with Planet Labs is around 50 petabytes, compared with about seven petabytes used for Pi and many large language models, and says satellite observations are only one dataset.

Q: Why does Sean White emphasize connection in AI development?

White argues that AI should be considered in terms of how it can connect people, rather than focusing only on fears of separation and loneliness. Conversation supports curiosity, surprise, insight, and continued learning. By giving understandable voices to complex systems, AI may help people relate more closely to themselves, organizational information, other species, and the natural world.

Summary & Key Takeaways

• Sean White argues that conversation is a powerful way to learn because it allows questions, surprises, and continued dialogue. Large language models make this interaction more accessible by learning human language, unlike earlier computers that required people to communicate through specialized programming languages such as 6502 assembly.

• White presents large language models as both computational systems and user interfaces. They can support personal reflection and make organizational data conversational, allowing users to investigate unusual patterns through follow-up questions. This approach could eventually extend to complex collective systems, including living cells, vehicles, rivers, animal species, and ecosystems.

• The talk imagines combining extensive environmental datasets to create an ongoing conversation with the planet. White suggests that AI should be developed with attention to connection rather than only fear and isolation. He closes by asking which collective intelligence people would choose to address and what they would ask it.