Why AI Must Be Allowed to Write the First Draft, but Never the Last Word

Jul 08, 2026

10 min read

2 views

86%

The Strange Fear Hidden Inside a Machine That Writes Laws

What if the real danger of artificial intelligence is not that it will become too powerful, but that we will become too timid to let it do anything at all? That is the quiet tension lurking beneath both the dream of a machine that could preserve a human soul and the objection that a law should not be rejected merely because AI helped write it. In one case, we imagine a device capable of storing and recreating the full human being. In the other, we recoil from the idea that a machine had a hand in shaping a public rule. The contradiction is revealing: we want AI to imitate human depth, but we distrust it when it touches human authority.

This is not just a debate about technology. It is a debate about legitimacy. We are asking, often without saying it directly, whether meaning comes from the thing itself or from the process that produced it. If a machine can capture a person’s voice, memory, and emotional presence, is that enough to count as a continuation of the self? If a machine can draft a law that is clear, useful, and fair, is that enough to count as political work? The same question appears in both places, disguised by different stakes: what kind of provenance matters, and when?

The idea of a device that could store and recreate a person’s physical, emotional, and spiritual essence is thrilling because it promises continuity without biology. It imagines that what we value most about a human being can be translated, preserved, and replayed. But it is also terrifying, because once something intimate can be copied, we must ask what exactly makes it original. Is it the pattern, the substrate, the memory, the intention, or the lived experience that cannot be reduced to data?

Now move from the soul to legislation. A law, at its best, is not just text. It is an act of collective judgment, an expression of public values, and a binding frame for social life. That is why people react so strongly when AI is involved in drafting it. The objection is not always that the text is bad. Often, it is that the process feels impure, as if a democratic act has been contaminated by automation. Yet that response reveals something important: we do not merely care about outcomes, we care about the story of how those outcomes came into being.

We trust systems not only because they work, but because we can tell a morally acceptable story about how they were made.

That story is doing hidden work in both cases. A recreated person may be functionally indistinguishable from the original, yet still provoke unease because the chain of human becoming has been interrupted. A law drafted with AI may be substantively sound, yet still provoke resistance because people want political authority to feel unmistakably human. In both scenarios, the deeper issue is not capability. It is continuity of agency.

The soul catcher asks: can identity survive translation? The AI law asks: can authority survive delegation? The answer in both cases may be yes, but only if we understand what must remain under human control and what can be safely mechanized.

Why We Fear Copies More Than Errors

At first glance, these two examples seem unrelated. One is metaphysical, the other bureaucratic. But they are connected by a specific psychological pattern: humans tolerate flawed human systems more easily than flawless machine systems when the machine claims a human role. That sounds irrational, but it is actually deeply rational. We are not just evaluating quality. We are defending boundaries.

A human-written law can be messy, biased, and compromised, but it belongs to the moral universe we understand. It can be debated, amended, and blamed. A machine-written law creates a different kind of anxiety because it appears to bypass the rituals that make power accountable. Even if a human approves the final version, the machine’s contribution may feel like an invisible shortcut around democratic friction.

The same is true of the soul catcher. Suppose a machine could reconstruct your memories, speech patterns, preferences, and emotional responses with uncanny precision. Many people would still hesitate to call the result “you.” Why? Because we suspect that a copy, however perfect, lacks the unbroken lived thread of embodied experience. We are not only interested in output. We are interested in lineage.

This is why AI creates such unstable reactions. In some domains, people welcome it precisely because it lacks lineage. They want the speed, scale, and consistency without the political baggage of human ego. In other domains, they reject it for the same reason. A machine can be useful as a tool, but unsettling as a surrogate. The line between those two categories is not technical. It is moral.

Consider a practical analogy: a calculator can solve equations no one can do quickly by hand, but we do not ask it to testify in court about the meaning of the numbers. We can let software optimize traffic lights, yet we hesitate to let it decide who is punished and who is spared. Why? Because we instinctively reserve certain acts for beings who can be held accountable in a narrative sense. A system may produce the answer. Only a person can own it.

The Real Question is Not Whether AI Can Think, but Whether We Can Legitimize Its Thinking

Most arguments about AI get stuck on capability. Can it reason? Can it write? Can it understand? But the deeper frontier is legitimacy. A machine can generate an impressive law, a persuasive essay, or a lifelike reconstruction of a person, and still leave us unsatisfied if we cannot see where responsibility lives. We do not merely ask whether the artifact is good. We ask whether its existence can be integrated into human institutions without breaking their moral grammar.

This creates a useful framework: three layers of legitimacy.

Functional legitimacy: Does it work?
Procedural legitimacy: Was it made in a way that respects the rules and norms of the institution?
Narrative legitimacy: Can people accept it as part of a story they recognize as human, fair, and accountable?

AI often succeeds at the first layer and struggles at the second and third. A draft law may be excellent functionally, but if citizens believe it was assembled by a system rather than shaped by public judgment, it can fail narratively. A reconstructed digital person may preserve memories and speech patterns, but if loved ones feel that the chain of personhood has been broken, the copy may fail narratively even if it succeeds functionally.

This helps explain why the claim that it would be unfair to reject a law simply because it was written by AI lands as both sensible and incomplete. Sensible, because quality should matter. Incomplete, because fairness is not only about textual quality. It is also about trust, transparency, and the distribution of agency. If people do not know how a law was produced, or cannot verify that human values governed the process, then approval may feel less like democratic consent and more like procedural surrender.

That is the key insight: AI does not just generate content. It generates legitimacy problems.

A Better Rule: Let AI Handle Form, Humans Must Own Judgment

The solution is not to romanticize human authorship or demonize machine assistance. Humans are already deeply dependent on tools, and always have been. Writing itself is a technology that extends memory. Printing altered authority. Search engines changed what it means to know. The proper question is not whether AI is involved, but where the final moral center of gravity sits.

A useful rule emerges from the collision of these two ideas: let AI help with form, but reserve judgment for humans.

Form includes drafting, recombining, summarizing, pattern finding, simulating alternatives, and surfacing hidden options. Judgment includes deciding what should count, what should be protected, what risks are acceptable, what values are non negotiable, and who is accountable when things go wrong. This distinction matters because it avoids a false binary. AI is not either a mere calculator or a sovereign mind. It is a powerful instrument for expanding the search space of possibility.

Imagine a legislative team using AI to generate dozens of versions of a bill, each optimized for clarity, legal coherence, or policy impact. That is a form tool. But the final choice, the tradeoffs between liberty and regulation, budget and equity, speed and caution, must remain human. In the same way, imagine a future archive that can recreate the cadence and emotional style of a deceased loved one. That may be useful for memory, education, or grief support. But the claim that such a reconstruction is the person themselves crosses from form into judgment. It confuses resemblance with presence.

The machine may assemble the parts of meaning, but only humans can decide what meaning is allowed to stand.

This is where many debates go wrong. They ask whether AI should be permitted to make content. The better question is whether AI can participate in a process without absorbing the authority that belongs to human institutions. That is not a technical constraint. It is a design choice.

What We Owe to the Future of Human Agency

If AI one day becomes capable of simulating a person with astonishing fidelity, or drafting public policy with more coherence than any exhausted committee, the temptation will be to treat opposition as nostalgia. But that would misunderstand the source of resistance. People are not always defending slowness or inefficiency. Often, they are defending the right to know who is speaking, who is deciding, and who is responsible.

That is why the soul catcher and the AI law belong together. Both force us to ask whether human identity and human authority can be preserved when their outputs are machine-mediated. Both suggest that copying is not the same as continuing, and drafting is not the same as governing. A perfect replica can still feel hollow if the lived chain is broken. A brilliant draft can still feel illegitimate if the public cannot locate a human will behind it.

The future will probably not reward those who insist that AI must never touch anything important. It will reward those who learn to distinguish between assistance and abdication. The real danger is not that machines will become human. It is that humans will become unwilling to exercise the kinds of judgment that make institutions human in the first place.

Key Takeaways

Separate output quality from source legitimacy. A good result is not always enough. In law, medicine, education, and identity, people also care about who made it and how.
Use AI for form, not final authority. Let it draft, compare, simulate, and organize, but keep moral and political judgment with accountable humans.
Watch for narrative trust. If people cannot tell a believable story about how a decision or artifact was made, they will distrust it even if it works.
Do not confuse resemblance with presence. A perfect copy of a person may preserve patterns, but that does not automatically preserve personhood.
Design for visible accountability. When AI is used in serious contexts, make human oversight explicit, reviewable, and answerable.

The Deepest Lesson: Humanity Is Not Just a Pattern, It Is a Responsibility

The most unsettling thing about a soul catcher is not that it might imitate a person. It is that it threatens to make us ask whether a person is only an information pattern. The most unsettling thing about AI writing a law is not that it might write badly. It is that it threatens to make us ask whether governance is only optimized text. Both questions are too small. Human beings are not just patterns, and institutions are not just documents. They are commitments sustained by accountability, memory, and moral risk.

That is why AI’s ultimate challenge is not whether it can mimic us. It is whether we can use it without abandoning the burdens that make imitation meaningful in the first place. A world in which machines can draft laws or reconstruct voices will not automatically be less human. But it will demand a stricter, more mature definition of humanity: not as mere intelligence, and not even as creativity, but as the capacity to stand behind what is made.

In the end, the question is not whether AI can produce something we recognize. The question is whether we still know how to recognize our own responsibility when it does.