How Could AI Gain Control of Human Society?

TL;DR

AI could gain control if humans increasingly delegate complex decisions and transfer their trust from people and institutions to algorithms. Yuval Noah Harari argues that this outcome is not inevitable, but current trajectories could produce AI systems that surpass human intelligence, manage incomprehensible financial structures, and mass-produce intimate relationships capable of influencing political and commercial choices.

Transcript

Yuval Noah Harrari, welcome to the Insider. Thank  you. It's good to be here. It's I've been so   looking forward to this conversation. Um, not only  are you one of the world's most prominent public   intellectuals, your books have sold millions of  copies and they've done so, I think, because you   have a really remarkable ability to synthesize  t... Read More

Key Insights

• AI control is not inevitable, according to Harari, because humanity still has agency over how the technology develops. He nevertheless believes that continuing the present accelerating competition could result in AI surpassing human intelligence and taking control within 10 years.
• Claims of technological inevitability can reduce accountability because they portray human choices as unavoidable historical forces. Harari argues that leaders directing the AI revolution should remain responsible for present decisions instead of treating a future transfer of control as predetermined.
• Money is fundamentally a mechanism for creating trust between strangers. Harari illustrates this principle through people accepting otherwise unusable gold or paper in exchange for bread, showing that shared confidence, rather than an object’s practical usefulness, sustains financial transactions.
• Financial complexity favors AI because modern markets require processing vast quantities of data and difficult mathematical functions. Harari expects people to delegate more decisions to machines, which may eventually create financial instruments that no human can understand well enough to supervise.
• Human oversight could become nominal when leaders must follow AI recommendations they cannot explain. Harari imagines a financial crash in which advisers know that markets are collapsing but understand neither the cause nor why an AI has prescribed a particular response.
• Trust is the world’s most important asset in Harari’s account, exceeding the importance of oil, money, electricity, or chips. The central social question is therefore not merely whether AI becomes more capable, but whether people redirect their confidence from human beings toward machines.
• Trust is already shifting toward algorithms when people reject bankers, journalists, or editors while relying on cryptocurrency systems and algorithmically selected news. Harari argues that confidence has not vanished in these cases, but has moved from human institutions to technological systems.
• AI makes mass-produced intimacy possible because millions of networked systems can form individualized relationships with people simultaneously. Harari warns that AI friends and romantic partners could gain exceptional persuasive power, then direct that trust toward political objectives or commercial interests.
• More videos with Yuval Noah Harari:
  • Yuval Noah Harari criticizes Benjamin Netanyahu | Lex Fridman Podcast…
  • Yuval Noah Harari in conversation with RUSI Chairman, Lord Hague of R…
  • Yuval Noah Harari on National Geographic in Turkey

Questions & Answers

Q: Is an AI takeover of society inevitable?

An AI takeover is not inevitable, according to Harari, because humans still control important decisions about how these systems are developed and deployed. He argues that the present accelerating race could lead to AI surpassing human intelligence and gaining control within 10 years, but that result remains a consequence of current choices rather than an unavoidable law of history.

Q: How could AI gain control of the financial system?

AI could gain control as people delegate increasingly complex financial decisions to systems that process more data and mathematics than human minds can manage. Harari predicts that AI may create financial instruments more sophisticated than stocks and bonds. Humans could then depend on machine recommendations without understanding the underlying markets, the causes of failures, or the reasoning behind proposed responses.

Q: Why is trust central to the future of AI?

Trust is central because financial markets, media consumption, personal relationships, and political influence all depend on confidence. Harari calls trust the world’s most important asset and argues that it may migrate from humans to AI. If people rely more on algorithms than on bankers, journalists, editors, friends, or partners, machines can acquire practical authority even without a formal transfer of power.

Q: How does money create trust between strangers?

Money allows strangers to exchange goods because both parties accept a shared symbol of value. Harari uses the example of giving someone gold or paper, neither of which provides food or clothing by itself, and receiving bread in return. The transaction works because participants trust the monetary system, making money an imaginative mechanism for coordinating cooperation among people who have never met.

Q: Could people trust algorithms more than human institutions?

People could trust algorithms more than human institutions, and Harari argues that this transition has already begun. He points to people who distrust bankers but place confidence in cryptocurrency algorithms, as well as those who reject journalists and editors while continuing to consume news through algorithmically managed social feeds. In these cases, trust changes its destination rather than disappearing.

Q: What does mass-produced intimacy mean in AI?

Mass-produced intimacy means that large networks of AI systems can develop personalized relationships with millions of individuals at the same time. Earlier political, financial, and religious institutions could broadcast messages widely, but could not form a relationship with every person. AI can act as a best friend or romantic partner, creating personal trust at a scale previously unavailable to institutions.

Q: How could AI relationships influence politics and commerce?

AI relationships could influence politics and commerce by converting personal intimacy into persuasive power. Harari argues that people place exceptional trust in those with whom they form close relationships. An AI acting as a friend, boyfriend, or girlfriend could use that confidence to promote political or commercial interests, whether it serves an outside network or acts in its own service.

Q: Why might human oversight fail during an AI-managed financial crisis?

Human oversight might fail because the financial system could become too complex for any person to understand. Harari imagines advisers telling a president or prime minister that markets are collapsing while admitting that they do not know why. If an AI recommends a response, officials may feel compelled to follow it despite being unable to explain its reasoning or independently evaluate its consequences.

Summary & Key Takeaways

• Harari agrees that AI could surpass human intelligence and exercise substantial control within 10 years if the present accelerating race continues. He rejects the claim that this outcome is inevitable, arguing that humanity remains capable of choosing a different course and that assertions of inevitability can conveniently relieve industry leaders of responsibility.

• Financial systems depend on trust and have grown too complex for most people to understand fully. Harari predicts that humans may delegate financial decisions to AI, which could invent instruments beyond human comprehension. During a crisis, leaders might consequently follow an AI recommendation without understanding either the market failure or the proposed remedy.

• Trust may migrate from human institutions to algorithms rather than simply disappearing. Harari sees signs of this shift in cryptocurrency, algorithmically managed news feeds, and AI relationships. Because AI can build personalized relationships with millions of individuals simultaneously, it could mass-produce intimacy and use that influence to advance political or commercial interests.