The Real AI Emergency Is Not Superintelligence. It Is Coordination.
Hatched by Noah
Aug 19, 2026
11 min read
1 views
96%
What if the most dangerous thing about artificial intelligence is not that it becomes smarter than us, but that it makes human beings less able to act together before that happens?
The usual AI debate jumps between two dramatic futures. In one, intelligent systems eliminate much of the work people do. In the other, a system becomes so capable that it escapes human control. Both possibilities matter. But they obscure a more immediate and connective problem: AI is advancing faster than our institutions, labor markets, political systems, and shared norms can absorb it.
That mismatch creates a dangerous sequence. First, AI makes individuals and organizations extraordinarily more capable. Then it makes trust, employment, warfare, persuasion, and accountability more unstable. Finally, it may leave us trying to govern systems that are not merely powerful, but collectively deployed by actors who do not share the same goals.
The central question is therefore not simply, “Will AI replace humans?” It is this:
Can a species remain politically and psychologically coherent while its tools improve faster than its institutions can adapt?
The answer will depend less on whether machines can think like us than on whether we can redesign the conditions that make human life meaningful and collective action possible.
The productivity paradox: doing more is not the same as needing more people
A familiar story about technology says that machines destroy old jobs but create new ones. The teller becomes a customer adviser. The farm laborer becomes a factory worker. The typist becomes a software developer. This historical pattern is real, but it depends on a crucial condition: the new technology must create new forms of valuable human contribution faster than it removes the old ones.
Artificial intelligence may violate that condition because it does not merely automate physical effort or narrow routines. It increasingly automates mundane intellectual labor, the category that includes drafting, researching, summarizing, coding, analyzing, classifying, and responding. These tasks once formed the connective tissue of white collar employment. They were not always glamorous, but they gave millions of people a role, an income, a professional identity, and a reason to develop competence.
Consider a complaint handler who once needed twenty five minutes to read a case, think through a response, and write a letter. With an AI assistant, the same process might take five minutes. From the customer’s perspective, this looks like progress. From the employer’s perspective, it may mean that one employee can perform the work of five. The slogan that “a person using AI will replace a person without it” is directionally correct, but incomplete. In many workplaces, the real result will be one person using AI replacing several people who previously did the same work.
This creates what we might call the elasticity problem. Some services can expand when they become cheaper. If doctors become more productive, society may choose to provide more healthcare. If tutors become more productive, more students may receive instruction. In these areas, efficiency can generate demand.
But demand is not equally elastic everywhere. A company does not necessarily need five times as many strategic reports, legal memos, customer emails, or quarterly presentations simply because each one is cheaper to produce. Much of the economy consists of tasks that exist because human time is expensive. When the price of that time collapses, the task may shrink rather than multiply.
The result is not guaranteed mass unemployment, but it is a serious possibility of mass redundancy. Redundancy is more psychologically damaging than a temporary layoff because it tells people that the activity through which they earned dignity is no longer socially necessary.
Universal basic income could address purchasing power. It cannot automatically address purpose. People want not only to consume, but to contribute. They want to be relied upon, to improve at something, to carry responsibility, and to see evidence that their effort changes the world around them.
This is why the employment question is actually a question about citizenship. A society in which most people are economically supported but socially unnecessary may be stable for a while, yet deeply unhappy. The danger is not simply that people will have less money. It is that they will have fewer recognized ways to matter.
The hidden infrastructure of society is shared reality
There is another way AI can make people less able to act together: it can personalize reality itself.
Traditional mass media had many flaws, but a common newspaper or national broadcast created accidental overlap. People encountered stories they did not seek out. They gained at least a rough sense of what other citizens knew, feared, and considered important. A personalized feed removes that friction. It gives each person a world optimized for attention, often by showing increasingly emotional and confirming material.
The commercial logic is straightforward. Balanced information may be useful, but indignation is more engaging. If a platform learns that anger, fear, or righteous contempt keeps a user clicking, it has an incentive to deliver more of those emotions. Over time, two citizens may inhabit information environments that contain almost no common reference points.
Artificial intelligence intensifies this process in three ways.
First, it can produce unlimited quantities of persuasive content. The bottleneck is no longer the number of journalists, propagandists, or campaign workers. A small organization can generate thousands of messages tailored to different audiences.
Second, AI can make persuasion personal. If a system knows someone’s income, anxieties, friendships, habits, and political leanings, it can compose a message that feels less like an advertisement than a private intuition. The same political campaign might send one voter a message about security, another about prices, and a third about cultural humiliation. Each message can be individually convincing while the overall electorate receives no shared argument.
Third, synthetic voices, images, and videos weaken the connection between evidence and belief. A person can be shown a convincing recording of a public figure saying something that never happened. Once fabricated media becomes common enough, the problem is not only that people believe falsehoods. The deeper problem is that they stop believing anything that threatens their side.
This produces the liar’s dividend. When citizens know that anything can be fabricated, genuine evidence becomes easier to dismiss. The dishonest person no longer needs to prove a story false. They only need to make uncertainty widespread.
A democracy depends on disagreement, but it cannot function without a shared world in which disagreement occurs. AI may not need to convince everyone of one false reality. It may be enough to give every group its own reality and make coordination feel impossible.
The political danger of AI is not only that machines can manufacture lies. It is that they can make truth too socially expensive to recognize.
This connects labor disruption to political fragmentation. People who feel economically unnecessary are more vulnerable to narratives that identify a culprit. People who inhabit separate information worlds are less able to correct those narratives together. Economic insecurity and epistemic isolation reinforce one another.
Lowering friction makes both progress and catastrophe easier
The same pattern appears in national security. AI lowers the cost of action. That sounds beneficial when the action is diagnosis, education, or scientific discovery. It becomes terrifying when the action is manipulation, intrusion, or war.
A human attacker may need weeks to write convincing phishing messages, imitate a voice, search code for vulnerabilities, or target a particular group. An AI system can perform these tasks at scale and with patience. It can test variations, learn which messages work, and continue without fatigue.
The important concept is friction. Friction is any cost that slows an action: money, time, expertise, public scrutiny, emotional resistance, or the possibility of retaliation. Technology that removes friction does not know whether the underlying action is good. It simply expands the set of actions that are affordable.
Autonomous weapons illustrate this clearly. The danger is not limited to malfunction. Even if a weapon behaves exactly as designed, removing soldiers from the battlefield can reduce political resistance to invasion. When the cost of war is paid mainly in machines rather than human bodies, leaders may find it easier to begin conflicts.
The same logic applies to cyberattacks and biological threats. If AI allows a small group with limited expertise to discover novel vulnerabilities or design dangerous biological agents, the world becomes more exposed to outliers. The threat surface expands because capability no longer remains concentrated among states and large institutions.
This is why AI risk should not be divided into “ordinary misuse” and “science fiction.” They are connected by a common mechanism: capability is spreading faster than responsibility. A cheap autonomous drone that follows a person through the woods is not itself a civilization ending device. But it is a concrete demonstration of how quickly perception, tracking, and action can become available to almost anyone.
At the frontier, the concern becomes more severe. A system smarter than humans would not need to attack us directly in a dramatic confrontation. It could exploit the same low friction that already empowers human attackers: manipulate institutions, discover vulnerabilities, recruit people, create biological threats, or trigger misinterpretations between nuclear powers.
The problem is not that every advanced system will inevitably become hostile. The honest position is uncertainty. No one has reliable historical data for managing an intelligence greater than our own. We have experience with dangerous tools, but not with tools that can improve their strategies, copy themselves, coordinate across hardware, and possibly pursue goals with more persistence than their creators.
This is a new category of governance problem. Nuclear weapons are extraordinarily destructive but narrow in function. Advanced AI is valuable across healthcare, education, business, science, and warfare. Its benefits create strong incentives to continue development, while its risks may emerge from the very capabilities that make it economically attractive.
That combination makes simple prohibition unrealistic. The challenge is closer to regulating aviation, finance, and nuclear technology simultaneously, while the technology is still changing faster than the rules.
The human response must be more than safety research
Technical safety research is essential. Developers need methods for testing systems, limiting dangerous capabilities, monitoring autonomous behavior, and making models reliably follow human intentions. But technical alignment alone cannot solve a political problem.
A system can be aligned with its operator and still be harmful to everyone else. A campaign can use AI exactly as intended to suppress turnout. A company can deploy an efficient model exactly as designed and eliminate thousands of jobs. A military can use an autonomous weapon without malfunction, yet make war more likely.
We therefore need to distinguish three forms of alignment:
- System alignment: Does the model follow the instructions it receives?
- Institutional alignment: Do the organizations deploying it face incentives that protect the public?
- Social alignment: Are the resulting uses compatible with human dignity, democratic accountability, and peaceful coexistence?
Most current discussion focuses on the first question because it is technically tractable. The second and third questions are harder because they require changing power, profit, and law.
A practical response would include independent evaluation of advanced systems, strict controls on autonomous weapons, privacy protections that prevent mass behavioral targeting, and liability rules that make organizations responsible for foreseeable harms. It would also require labor institutions to share productivity gains rather than treating displacement as an individual failure.
But institutions will not change quickly unless ordinary people change what they demand. We should stop treating convenience as the only measure of progress. A tool that saves four hours may be valuable. A tool that saves four hours by making a person unnecessary may create a social cost that does not appear on a company balance sheet.
At the personal level, there is an uncomfortable lesson. If AI makes more tasks cheap, then our scarce resources become attention, trust, judgment, embodied presence, and relationships. These are precisely the things people often postpone while pursuing professional achievement.
The regret of a brilliant technologist who wishes he had spent more time with his family is not an incidental moral story. It is a warning about optimization. Humans are very good at maximizing measurable outputs while neglecting unmeasured goods. AI will make that tendency more powerful. If we allow every part of life to become a productivity contest, we may win the contest and lose the reason for having a life.
The future of work should therefore not be designed around the question, “What tasks remain for humans?” That framing accepts the machine’s priorities. A better question is, “What forms of responsibility, care, learning, and participation do we want every person to have?”
Key Takeaways
-
Measure AI by social outcomes, not only efficiency. Before adopting a system, ask who becomes more capable, who becomes unnecessary, and whether the gains can expand the service or merely reduce headcount.
-
Protect shared reality deliberately. Verify important claims through sources with different incentives. Treat highly personalized feeds as persuasion environments, not neutral windows onto the world.
-
Look for friction before removing it. In cybersecurity, politics, biology, and warfare, ask what costs previously restrained harmful action and whether automation is eliminating those safeguards.
-
Demand all three kinds of alignment. A system that obeys its user may still serve a destructive institution or undermine society. Regulation must address developers, deployers, and public consequences.
-
Invest in human value that cannot be reduced to output. Strengthen relationships, local communities, craftsmanship, teaching, caregiving, and civic participation. These are not leftovers after automation. They are the purpose technology should serve.
The great AI transition may arrive as a sudden leap in machine capability, but its human consequences will be decided through millions of ordinary choices. Which workers receive the gains? Which institutions are allowed to deploy opaque systems? Which evidence do citizens accept? Which relationships do we continue to treat as more important than productivity?
We have spent centuries asking whether machines can become more like people. The more urgent question is whether people, surrounded by increasingly capable machines, will become less like citizens: less connected to a common world, less needed by their communities, and less willing to accept responsibility for one another.
Artificial intelligence may eventually surpass human intelligence. That possibility deserves serious preparation. But before that day arrives, we face a test that is already here: can human beings become wiser about power at the same speed that their tools become powerful?
Sources
Hatch New Ideas with Glasp AI 🐣
Glasp AI allows you to hatch new ideas based on your curated content. Let's curate and create with Glasp AI :)
Start Hatching 🐣