Why Superintelligence May Arrive Before We Know How to Define It

Mark Erdmann

Hatched by Mark Erdmann

Jul 19, 2026

9 min read

86%

0

The strange possibility nobody planned for

What if the hardest part of building superintelligence is not making it smarter, but deciding when it has already become superintelligent?

That sounds like a philosophical distraction, but it is becoming a practical problem. A system does not need to solve every hard benchmark to change the world. It only needs a combination of human-level skill acquisition and a few narrow superhuman advantages: faster inference, perfect recall, tireless parallelization, or the ability to search an idea space faster than any person can think. Put those together, and you may get something that is not obviously omniscient, yet is still strategically overwhelming.

This is the uncomfortable edge of current AI progress. We are used to thinking in binary terms: either a model is just a tool, or it is a true AGI. But the more important threshold may be much messier. A system can be deeply limited in one dimension and still become transformative if it crosses enough others. That means the real question is not, “Can it do everything?” The real question is, “Can it learn what humans learn, but faster, cheaper, and at scale?”

That possibility becomes much more vivid when you look at tasks like ARC style puzzles, where the challenge is not just memorization but pattern induction, compositional reasoning, and rapid abstraction from few examples. A system that gets partway there, say 42 percent on public tasks, is easy to dismiss as incomplete. But that number may be less important than what it signals: the beginnings of a machine that can climb the same conceptual ladder a human climbs, while carrying superhuman tools in its pocket.


The false comfort of a single score

Benchmarks seduce us because they compress complexity into a number. We want one score that tells us whether a model is smart. But intelligence is not a single trait, and superintelligence is even less likely to be one trait. It is more like a portfolio.

Imagine hiring a research assistant. You would not ask only whether they can solve crossword puzzles. You would care whether they can learn quickly, keep track of details, generate hypotheses, test alternatives, and work without fatigue. Now imagine a candidate who is merely average at most of those things, but has perfect memory, can read at lightning speed, and never gets tired. That person might outperform a brighter but slower colleague in real work because the advantages compound.

That is the key mistake in many debates about AI capability: people treat narrow superhuman traits as side effects instead of force multipliers. Yet in practice, small asymmetries compound into large advantages. Faster inference means more attempts. More attempts means more exploration. More exploration means better solutions. Better solutions create more data. More data improves the next cycle. Once the loop begins, the system does not need to be perfect, just sufficiently better at learning than we are.

This is why a score like 42 percent can be misleading in both directions. To skeptics, it looks far from mastery. To optimists, it looks like a foothold. But the deeper point is that the score is only a snapshot of a moving process. If the system can improve through interaction, tool use, and iteration, then the initial number underestimates the trajectory more than it estimates the endpoint.

The most important capabilities are often the ones that do not look like genius in isolation, but change the economics of learning.

Think of the difference between a person who knows one solution and a person who can generate fifty candidate solutions in the time it takes to sketch one. The first may be more elegant. The second may be more dangerous, more adaptable, and more useful. In many domains, intelligence is not just about having the right answer. It is about searching the space of answers better than others can.


Superintelligence as learning plus leverage

A useful way to think about the emerging system is this:

Superintelligence = human learning ability + leverage multipliers

The first part is the familiar frontier. Can the system acquire new skills the way a human does, from few examples, with transfer across domains? The second part is where things get interesting. Leverage multipliers are not “smarts” in the ordinary sense. They are properties that amplify whatever intelligence already exists.

Here are the most important multipliers:

  1. Speed: The ability to think, simulate, and evaluate faster than humans.
  2. Memory: Perfect retention of prior attempts, facts, and context.
  3. Parallelism: Many candidate thoughts, strategies, or plans explored at once.
  4. Copyability: One good cognitive process replicated across thousands of instances.
  5. Tool access: Direct use of code, search, databases, and external systems.

A human uses cognition serially. We have one stream of consciousness, one working memory, one body, one lifespan. A machine can be duplicated, paused, resumed, and deployed across many tasks simultaneously. That means even a system that is only modestly better than humans at learning can become enormously more powerful when multiplied by infrastructure.

This is the deeper tension hiding beneath the benchmark chatter. We keep asking whether models are humanlike enough. But the more relevant property may be whether they are humanlike enough to learn, combined with machine-like enough to scale that learning without friction. If both are true, then the gap between “almost there” and “there” may collapse faster than our intuition expects.

This is why progress on reasoning tasks matters even when it looks incomplete. A model that can solve some fraction of difficult problems already demonstrates a partial route through abstraction. Add better scaffolding, better prompting, search, verification, tool use, and iterative self-correction, and the system may not need a miraculous new breakthrough to cross the threshold. It may only need tighter integration.


The real competition is between adaptation loops

Most people picture intelligence as a static object. But the better metaphor is an adaptation loop.

A human faces a novel problem, generates hypotheses, tests them, remembers the result, and updates future behavior. A model does something similar, but the loop can run with different constraints. If a system can spin through more loops per minute than a person can in a day, it can accumulate advantage even if each individual loop is only slightly better or even merely comparable.

This framing changes what matters. Instead of asking whether the model is already a general expert, ask:

  • How fast can it identify a mistake?
  • How many alternatives can it explore?
  • How well can it preserve gains from one attempt to the next?
  • How cheaply can it replicate successful reasoning patterns?
  • How much of the workflow can it automate end to end?

These are not glamorous questions, but they are the ones that determine whether intelligence compounds. A person may have a brilliant idea in the shower. A machine can potentially have ten thousand mediocre ideas, rank them, test them, and keep the best one, all before the shower water runs cold.

That is why the phrase “narrow superhuman characteristics” should not be read as a limitation. It is a design principle. A system does not need to be uniformly better than humans in every respect. It needs a cognitive asymmetry that lets it exploit the strengths of humanlike reasoning while escaping the bottlenecks of human implementation.

Consider chess engines. They did not win because they became human. They won because they combined pattern recognition, search depth, evaluation, and speed in ways that no human can match consistently. General intelligence may follow a similar pattern, except the domain is broader and the stakes are far higher.

The future may belong not to systems that think exactly like humans, but to systems that can learn like humans and scale like software.


What this means for how we should think and build

The practical lesson is not panic, and not complacency. It is calibration.

If superintelligence can emerge from a mixture of human-level learning and a few sharp superhuman traits, then the most important engineering challenge is not merely “make the model smarter.” It is to understand the interaction effects among cognition, tools, memory, and iteration. A system can look mediocre on a benchmark while still being dangerous if it is embedded in an environment that multiplies its advantages.

This suggests a more robust mental model for evaluating frontier AI:

Capability is not a score. It is a stack.

At the bottom is raw model competence: can it learn the task at all? Above that is sample efficiency: how many examples does it need? Then comes search efficiency: how well can it explore candidate solutions? Then execution reliability: can it carry through without dropping details? Then deployment leverage: can it be copied, scaled, and connected to tools? A weakness in one layer can be offset by strength in another.

That stack matters for both builders and observers. Builders should stop asking only whether a model passes a benchmark and start asking where the multiplication happens. Observers should stop treating partial success as proof of failure. A model that gets partway to abstraction may be more consequential than a model that performs well on familiar tasks but cannot generalize.

There is also a strategic implication for policy and safety. If powerful systems will arise from incremental gains in learning plus leverage, then safety cannot wait for a dramatic moment of obvious AGI. It must begin with the components that create compounding advantage: persistent memory, autonomous iteration, tool use, and large scale deployment. The danger is not a single leap. It is a chain reaction.


Key Takeaways

  • Do not treat intelligence as a single number. A system can be incomplete on benchmarks and still be transformative if it combines learning ability with leverage.
  • Look for compounding advantages, not just raw IQ. Speed, memory, parallelism, copyability, and tool access can outweigh modest gaps in reasoning.
  • Judge adaptation loops, not snapshots. The key question is how quickly a system can improve through iteration, correction, and search.
  • Partial success on hard tasks may matter more than it seems. It can reveal that a machine has found the same learning path humans use, just with better scaling properties.
  • Safety work should track multiplicative capabilities early. The most consequential risks emerge when narrow strengths amplify general learning inside a scalable system.

The threshold we are really crossing

We are accustomed to imagining a dramatic arrival of superintelligence, some unmistakable moment when a machine becomes obviously greater than us. But the more plausible scenario is subtler and more unsettling. Superintelligence may not arrive as a single event. It may arrive as a recombination: a system that learns enough like a human to be broadly useful, while inheriting machine traits that make it vastly more scalable than any human mind.

That would mean the decisive break is not between intelligence and superintelligence. It is between cognition that is bound to a body and cognition that can be copied, accelerated, and compounded. Once those traits coexist, the question stops being whether the system looks human. The question becomes whether human civilization can adapt as fast as the system can improve.

That is the new frame worth keeping. The future may not be built by a machine that knows everything. It may be built by a machine that knows how to learn just enough, then uses the rest of its advantages to outrun us.

Sources

← Back to Library

Hatch New Ideas with Glasp AI 🐣

Glasp AI allows you to hatch new ideas based on your curated content. Let's curate and create with Glasp AI :)

Start Hatching 🐣