When Rules Become Software, Who Gets to Make the Law?

What if the most important question about artificial intelligence is not what it can do, but who gets to decide what it is allowed to do?

That question appears in two settings that seem unrelated. In one, a proposed government remedy would place a powerful technical committee inside the machinery of online search, giving appointed officials substantial influence over how a service used by billions of people operates. In the other, a global law firm uses generative AI to inspect employees’ unstructured communications and identify possible violations of anti bribery rules before regulators or prosecutors do.

One looks like public regulation. The other looks like private enterprise. Yet both are expressions of the same profound shift: rules are moving from documents and institutions into software that continuously interprets, ranks, flags, and acts.

This shift creates an uncomfortable paradox. We increasingly need systems that can anticipate harm rather than merely punish it after the fact. But the more anticipation we automate, the more authority we quietly delegate to whoever designs the system, selects its training material, and decides what counts as a meaningful signal.

The central challenge of the AI era is therefore not simply innovation versus regulation. It is the governance of embedded judgment.

The rulebook is becoming a machine

For most of modern history, law and policy operated as external constraints. A statute sat in a book. A regulator issued guidance. A court interpreted disputed language after an event had occurred. A company then tried to translate those rules into policies, training sessions, audits, and employee behavior.

Artificial intelligence changes the direction of travel. Instead of asking people to remember a rule, organizations can build systems that watch for patterns associated with violating it. Instead of waiting for a complaint, a regulator, or a lawsuit, a company can scan communications and surface suspicious behavior in advance. The rule is no longer merely something people consult. It becomes part of the environment in which they work.

Imagine the difference between a sign that says “Drive carefully” and a car that continuously monitors speed, road conditions, driver attention, and nearby traffic. The sign informs. The car intervenes, or at least produces a warning. It turns an abstract norm into an operational behavior.

That is what a proactive compliance tool promises. Statutes such as the Foreign Corrupt Practices Act are not simple checklists. Their application depends on context, intent, industry conventions, geography, intermediaries, and patterns of conduct spread across many messages. A system trained with the participation of experienced legal professionals can search that complexity at a scale no human review team could match.

The benefit is obvious. A company may discover a risky payment arrangement, euphemistic language, or a suspicious relationship before the conduct matures into an investigation. Yet the system does more than find violations. It also defines what deserves attention. Its categories, thresholds, and escalation paths become a practical interpretation of the law.

When judgment is embedded in software, the people who configure the system become lawmakers in everything but name.

This is true even when the system has no formal legal authority. A flagged message can trigger an inquiry. An inquiry can alter a career. A pattern of warnings can change how employees communicate. The system may not convict anyone, but it can reshape behavior long before a court or regulator enters the picture.

The same dilemma appears in search

Search systems also convert values into infrastructure. They decide which information is visible, which sources appear authoritative, which results are promoted, and which forms of content are treated as relevant. Users experience these decisions as an interface, not as a policy debate.

That is why a proposal to place a government appointed technical committee in a position to influence search is more consequential than a dispute about corporate governance. It raises a question about whether public authorities should have an ongoing role in the design and operation of a complex information system used across markets and borders.

The concern is not that technical oversight is always illegitimate. Complex systems often require oversight, especially when failures can affect competition, privacy, safety, or access to information. The deeper concern is operational sovereignty: who has the continuing power to alter the rules by which a system sorts the world?

A committee with broad technical influence might begin with a narrow mandate. Over time, however, its decisions could affect ranking criteria, product design, data use, quality standards, or the balance between competing social objectives. Every intervention would look technical, but each would carry a political judgment. Should a system prioritize freshness or reliability? Local relevance or global consistency? Commercial usefulness or public interest? Speed or caution?

There is no purely technical answer to these questions. A committee may be well intentioned and staffed by experts, yet still lack the accountability, transparency, and feedback mechanisms needed to govern a system at planetary scale. Technical competence is not the same thing as legitimate authority.

The same distinction applies inside a law firm or corporation. Subject matter experts can train a compliance model on the relevant statutes and help identify meaningful patterns. That expertise is essential. But expertise alone does not settle how aggressively the model should flag ambiguous language, how employees should be notified, who can inspect the records, or what happens when the system is wrong.

In both cases, the hard problem is not adding intelligence to a system. It is deciding whose interpretation becomes the default behavior of the system.

The net effect is not neutral

There is an appealing argument that automation will destroy some work but create more work elsewhere. A law firm may use AI to make routine analysis more efficient, yet the resulting lower cost could generate new legal matters. Companies might monitor more transactions, enter more regulated markets, or seek advice on risks they previously could not afford to examine. Efficiency can expand the addressable universe of demand.

This is a useful way to think about technological change. But it needs a missing variable: the distribution of power created by the new system.

Suppose an AI tool reduces the cost of reviewing compliance data by 90 percent. A company can now inspect every employee communication rather than a small sample. That may create new legal work, as more potential issues are discovered. It may also produce a culture in which employees write defensively, avoid candid discussion, or route ordinary decisions through informal channels that are harder to monitor. The system has increased detection while potentially degrading communication.

Or suppose a search platform becomes subject to continuous technical intervention by an external committee. The intention may be to prevent abuse or protect competition. But if the system becomes slower to improve, less coherent across countries, or more vulnerable to political pressure, users pay the cost through worse information access. A remedy designed to limit one form of power can create another form of power: the power to delay, constrain, or redirect technological evolution.

This suggests a more complete version of the net effect theory:

The net effect of AI is the sum of efficiency gains, newly created demand, and newly distributed authority.

The first two are easy to measure. The third is often ignored.

A useful evaluation framework asks four questions:

1. What work disappears? This includes routine review, manual search, and repetitive interpretation.
2. What new work appears? This includes newly affordable audits, investigations, product categories, and legal questions.
3. What behavior changes? People adapt to what systems reward, flag, or make costly.
4. Who gains discretionary power? This may be a vendor, a regulator, a technical committee, a compliance team, or the people who control model settings.

The fourth question is the one most likely to be omitted from a conventional productivity analysis. Yet it often determines whether the technology is trusted.

From prevention to preemption

Proactive compliance illustrates the attraction of acting before harm occurs. Waiting for a bribery scheme to become visible can be enormously expensive. Early detection may protect employees, shareholders, customers, and the public. In high consequence domains, prevention is not merely efficient. It is morally compelling.

But prevention can easily become preemption. A system that identifies risk is not the same as a system that establishes wrongdoing. The closer an alert comes to triggering punishment automatically, the more dangerous false positives become.

This is where a practical distinction matters: detection, interpretation, and intervention should not be collapsed into one automated step.

A robust architecture separates them:

• Detection identifies unusual or potentially relevant patterns.
• Interpretation places those patterns in legal, organizational, and human context.
• Intervention determines what response is proportionate, reversible, and fair.

For example, a message mentioning a government official and a payment may deserve review. It should not, by itself, determine that an employee violated anti bribery law. A ranking anomaly may deserve investigation. It should not automatically establish that a search system is unfair or that a particular result must be removed.

This separation is more than a technical safeguard. It preserves the difference between a machine’s confidence and a human institution’s judgment. Models are good at finding correlations across enormous volumes of information. Institutions are responsible for explaining decisions, hearing objections, and absorbing consequences.

The same principle should guide public oversight of search. A technical committee might be able to audit, test, and report on system behavior without receiving open ended authority to redesign the service. Oversight can be powerful without becoming continuous operational control. The goal should be to make systems inspectable and contestable, not to turn every technical decision into a command from an external administrator.

The design principle that connects both worlds

The best response to this new era is neither unrestricted automation nor blanket human control. It is contestable infrastructure.

A contestable system has five properties.

First, its consequential decisions are legible. People can understand why a communication was flagged or why a result was ranked in a particular way, at least well enough to challenge the outcome.

Second, its boundaries are explicit. A model should not quietly expand from identifying possible compliance concerns to monitoring every aspect of employee conduct. A regulator should not quietly expand from auditing a platform to directing its product roadmap.

Third, its interventions are proportional. A low confidence signal should produce a request for review, not an irreversible penalty. A suspected design problem should produce a measured investigation, not immediate control of the entire system.

Fourth, its decisions are reversible. False positives are inevitable. The question is whether a person can correct them without disproportionate cost.

Fifth, responsibility remains visible. If everyone involved can say that the model, committee, vendor, or process made the decision, accountability has disappeared. Human beings and institutions must retain named responsibility for the consequences of automated judgment.

These principles apply to both corporate compliance and public technology governance because both are becoming forms of continuous administration. The system is no longer used occasionally to answer a question. It is constantly shaping what questions are asked, what risks are noticed, and what options appear available.

Key Takeaways

• Treat AI systems as policy systems, not merely productivity tools. Ask what norms they encode, whose assumptions they reflect, and how their outputs change behavior.
• Measure authority alongside efficiency. When evaluating an AI initiative, identify who gains the power to flag, rank, delay, approve, or escalate decisions.
• Separate detection from punishment. Use AI to surface patterns, but preserve human review for interpretation and consequential intervention.
• Design for contestability from the beginning. Build explanations, appeal channels, audit logs, clear boundaries, and reversible actions into the system rather than adding them after a scandal.
• Prefer oversight that makes systems inspectable over control that makes them centrally directed. The ability to test a complex technology is not the same as the legitimacy to operate it.

The most important AI governance question is therefore not whether a system is accurate in the abstract. Accuracy depends on the task, the context, and the cost of being wrong. A compliance model that catches more risks but accuses innocent employees may be worse than a less sensitive system with stronger review. A search remedy that appears principled but makes information less reliable may weaken the public interest it was meant to serve.

The deeper question is: Can people affected by an intelligent system understand, challenge, and correct its judgments?

That question reframes the debate. It moves us beyond the stale opposition between innovation and regulation, as if one must defeat the other. We need innovation capable of operating under legitimate constraints, and regulation capable of constraining systems without becoming an unaccountable system of its own.

AI will create new legal work, new markets, and new forms of oversight. But its most consequential product may be less visible: the gradual relocation of judgment into the infrastructure of everyday life. Once that happens, governance is no longer something applied to technology from the outside. Governance is the technology.

The institutions that understand this will not ask only whether an AI system works. They will ask what kind of world its working makes possible, who is empowered within that world, and whether those subject to its decisions still have a meaningful way to say no.