Yuval Noah Harari on AI as an Alien Agent and Trust

TL;DR

AI is the first technology that acts as an agent rather than a tool: it can make decisions and invent new ideas without human help. Yuval Noah Harari argues that its main danger is that it is alien, not that it is evil. AI leaders say they cannot slow down because they cannot trust rival humans, yet they claim they will be able to trust superintelligent AI.

Transcript

Yuval, thank you so much for coming to Keio University. Thank you for the invitation. It's an honor to be here. We only have 30 minutes. So I'm going to bombard you with questions. Absolutely. So first question to you. When I assume the position of a president of universities, let's say four years ago, the university library set up to set up a self... Read More

Key Insights

• AI is the first technology humans have created that is an agent rather than a tool. A nuclear reactor cannot make decisions or design the next reactor, but AI can make decisions and invent new ideas, weapons and medicines without human intervention.
• The key problem with AI is not that it is evil but that it is alien. Harari argues it may surpass human intelligence in many ways while never reaching 'human level' intelligence, just as airplanes fly but never fly like birds.
• AI moves far faster than biotechnology. A generation in human biotechnology may take 20 or 30 years to evaluate, while in AI a generation could take just a few days, so digital evolution runs millions of times faster than organic evolution.
• Nuclear war is a huge threat that everyone understands, because it has no upside and no winners. AI is harder to handle because it has a large positive potential, and people find its threats difficult to grasp.
• Harari describes a paradox of trust at the heart of the AI revolution. Leaders say they must race ahead because they cannot trust rival companies or countries, yet they say they will be able to trust the superintelligent AI they are building.
• Humans have thousands of years of experience learning to trust each other. Small bands of a few dozen people grew into nations of millions and a global trade network connecting 8 billion people, but humanity has zero experience with superintelligent AI.
• Expected timelines for artificial general intelligence have shortened sharply. Ray Kurzweil's forecast of AGI by 2029 once seemed exaggerated to Harari, yet Kurzweil is now considered conservative compared with developers who speak of one to five years.
• Harari says hopes for a global agreement on managing AI risk now look naive. In his view, since the last US elections, the hope of reaching such an agreement is gone.

Questions & Answers

Q: Why does Yuval Noah Harari say AI is an agent and not a tool?

Harari says every earlier technology, including the atom bomb and nuclear reactors, was a tool: humans decided how to use it, and the machine could make no decisions or invent anything new. AI is the first technology that can make decisions by itself and invent new ideas. An AI weapon can decide what to bomb, and AI can invent new weapons, military strategies, medicines and cancer treatments without human intervention.

Q: What does Harari mean when he calls AI an alien intelligence?

Harari argues that the key problem with AI is not that it is evil but that it is alien. It is not organic and not like human beings. Asked when AI will reach human level intelligence, he answers never, comparing it to asking when airplanes will reach bird level flight. AI may surpass human intelligence, but it will stay alien, which makes its development and consequences hard to predict.

Q: Why did Harari focus on AI instead of biotechnology or nuclear war in Nexus?

Harari says AI moves much faster than biotechnology: judging the effects of a change to a human genome could take 20 to 40 years, while an AI generation could take days. Nuclear war is a huge danger, but it has no upside and no supporters, so everyone understands it. AI is harder to deal with because it has large positive potential and its threats are difficult for people to grasp.

Q: What is the paradox of trust in the AI revolution?

Harari asks AI leaders and politicians why they move so fast. They say they understand the risks, up to human extinction, but cannot slow down because competitors in other companies or countries will not slow down, and the most ruthless would win. When he asks whether they could trust the superintelligent AI they build, they say yes. He calls this close to insanity: they distrust humans but expect to trust AI.

Q: Why does Harari say humans are better prepared to trust each other than to trust AI?

Humans have thousands of years of experience with other humans. We understand human psychology, motivation and the craving for power, and we have built mechanisms to check the pursuit of power. Early humans lived in bands of a few dozen people, while today nations of millions trust each other and global trade connects 8 billion people. With superintelligent AI we have no experience, and no idea what goals or tricks it might develop.

Q: How has Harari's view of AI changed over the past ten years?

Harari says that when he wrote about the future around 2015, few people outside computer science departments, companies like Google and Microsoft, and philosophers such as Nick Bostrom were talking about AI. It felt like speculation about something hundreds of years away. Now AI is all around us, development is moving at an astounding pace, and the hopes of regulating it now look extremely naive to him.

Q: When do experts predict artificial general intelligence will arrive, according to Harari?

Harari recalls that Ray Kurzweil predicted artificial general intelligence, AI able to surpass humans in all fields rather than narrow ones like chess or driving, by 2029. Harari once thought this was an exaggeration. Kurzweil still says 2029, but he is now considered one of the more conservative thinkers, since people leading AI development in the US and China talk about AGI within one to five years.

Q: Why does Harari compare AI to a wave of immigration?

Harari says AI is not one single big computer but millions of superintelligent AIs interacting with each other and with millions of humans. He compares it to a wave of immigration sweeping over the world. He notes that populist parties fear immigrants will take jobs and change the culture, and argues that AI is a wave of millions of alien immigrants that will take people's jobs.

Summary & Key Takeaways

• Speaking with the president of Keio University, Yuval Noah Harari explains why his book Nexus focuses on information networks and AI rather than nuclear war or biotechnology. AI moves far faster than biotechnology, and unlike nuclear weapons, which everyone understands are dangerous, AI has a large upside that makes its risks hard to grasp.

• Harari says AI is an alien intelligence and an agent, not a tool. It can make decisions and invent new medicines, weapons and strategies without humans. He calls this possibly the biggest challenge in human history, because for thousands of years nothing on Earth could compete with human intelligence.

• Looking back on his writing about the future around 2015, Harari notes that AI then seemed centuries away, while forecasts of AGI have since shrunk to a few years. He describes a paradox of trust: leaders race ahead because they distrust rival humans, yet expect to trust superintelligent AI, something humanity has no experience with.