What Is a Superforecaster?
A superforecaster is someone whose probability forecasts have been shown, across many scored questions, to be consistently more accurate than those of almost everyone else forecasting the same questions. The term comes from Philip Tetlock's Good Judgment Project, one of the research teams in a U.S. government forecasting tournament. Each year the project gave the title to its top 2 percent of forecasters.
Superforecasting is the method those people shared: break a question into parts, start from the base rate, put a precise number on your belief, and update it in small steps. Tetlock and journalist Dan Gardner lay it out in Superforecasting: The Art and Science of Prediction (2015), and most of it can be learned.
The tournament itself was a strange idea. In 2011 the Intelligence Advanced Research Projects Activity (IARPA), a research arm of the U.S. intelligence community, set out to learn whether anyone could forecast world events well, so it invited university teams to compete. Each team recruited volunteers to answer hard geopolitical questions with numeric probabilities: Will this country default? Will these two nations clash? Will this leader still hold power in six months?
Tetlock and Barbara Mellers ran the Good Judgment team. Over four years, thousands of online volunteers made more than a million forecasts across roughly 500 questions, and the results embarrassed the professionals. Good Judgment beat the tournament's official control group by 60 percent in year one and by 78 percent in year two. It also beat rival university teams, Michigan and MIT among them, by 30 to 70 percent. It even beat the intelligence community's own internal prediction market, whose analysts could see classified data: over 139 questions in 2013 and 2014, Good Judgment's best methods were 34.7 percent more accurate.
The project put each year's superforecasters on teams together and kept scoring them. If their records had been luck, they should have slid back toward the average. Most didn't. Roughly 70 percent kept their status the following year, and across all forecasters, accuracy in one year correlated about 0.65 with accuracy in the next.
One of the first superforecasters, a retired computer programmer named Doug Lorch, made roughly a thousand forecasts in his first year and finished fifth of about 2,800 competitors, with a Brier score of 0.22 (lower is better; more on Brier scores below). The next year, on a superforecaster team, he scored 0.14, the best in the project. He wasn't a spy or a quant. He was a hobbyist who'd found a system that worked.
The name has outlived the tournament. Good Judgment Inc., the project's commercial spin-off, still uses it (written Superforecaster®) for its professional forecasters, and AI researchers now treat superforecasters as the human benchmark that forecasting models are scored against.
Most people read the book to get better at predicting markets or politics. This guide treats it as a training manual for judgment, because thinking clearly about an uncertain future is the skill you use every time you decide what to believe, which sources to trust, and what to do with what you learn.
Why Most Expert Predictions Fail
Tetlock earned the right to write this book the hard way. His earlier work, Expert Political Judgment (2005), tracked 284 professional pundits and analysts making roughly 28,000 predictions over about twenty years. The famous verdict: the average expert was roughly as accurate as "a dart-throwing chimpanzee." Worse, the more famous the expert, the less accurate they tended to be, because confident, sweeping stories get you on television while careful hedging does not.
But the averages hid a split. Borrowing Isaiah Berlin's metaphor, Tetlock sorted forecasters into hedgehogs and foxes. The hedgehog knows one big thing and forces every question through a single grand theory. The fox knows many small things, distrusts big theories, and stitches together evidence from lots of sources. Foxes consistently forecast better. Hedgehogs made bolder predictions and got more attention, but their one big idea kept steering them into confident errors.
The lesson isn't that expertise is useless. It's that being knowledgeable and being well-calibrated are different skills. Knowing a lot about a topic tells you what could happen. It doesn't automatically tell you how likely each outcome is, and likelihood is what a forecast is about. That gap is why smart, informed people are so often confidently wrong, a pattern Daniel Kahneman spent a career documenting (see how to apply Thinking, Fast and Slow).
What Actually Makes a Superforecaster
The obvious guess is intelligence. Superforecasters did score higher than average on tests of intelligence and knowledge, roughly in the top 20 percent, but they weren't outliers. Plenty of very smart people forecast poorly, and plenty of merely bright ones excelled. IQ set a floor, not a ceiling.
What separated the best was a habit of mind Tetlock calls "perpetual beta," borrowing the software term for a product that's never finished and always being improved. Superforecasters treat every belief as a draft. They actively hunt for reasons they might be wrong, they revise in small steps as evidence trickles in, and they run honest postmortems on their misses. Tetlock writes that this commitment to belief updating and self-improvement is the strongest predictor of rising into the superforecaster ranks, roughly three times as powerful as its closest rival, intelligence. Adam Grant built a whole book, Think Again, around that habit.
The people themselves were unremarkable on paper. Bill Flack, one of the standout forecasters, was a former U.S. Department of Agriculture employee living in Kearney, Nebraska, and an avid bird-watcher. Mary Simpson, an economist who also became a superforecaster, has said she "had totally missed the 2007-2008 financial crash," and she came to the tournament "very skeptical of the ability of anyone to forecast in certain arenas." Getting good at something you once missed badly is Carol Dweck's growth mindset, pointed at a single trainable skill.
Much of this can be taught, and fast. In a randomized experiment, about an hour of training on base rates, calibration, and common biases improved forecasters' accuracy by 6 to 11 percent, and trained forecasters beat the untrained controls in each of the tournament's four years. Teams helped too: on average they were 23 percent more accurate than individuals, and teams of superforecasters beat prediction markets by 15 to 30 percent. The reassuring implication of the whole book is that superforecasting is learned behavior, not a personality you're stuck without.
Fermi-ize the Question and Start From the Outside
Faced with a big, vague question, most people reach for their gut and a headline. Superforecasters do the opposite. They break the question into smaller sub-questions, each easier to reason about, a technique Tetlock calls "Fermi-izing" after the physicist Enrico Fermi, who was famous for estimating hard quantities (like the number of piano tuners in Chicago) by decomposing them. The point of breaking a problem apart is to separate what you know from what you're guessing, or as Tetlock puts it, to "flush ignorance into the open."
The next move is counterintuitive: start from the outside, not the inside. The inside view is the specific story in front of you, all its vivid details. The outside view asks a colder question: how often do things like this happen in situations like this? That base rate is your anchor. If you want to forecast whether a startup will still exist in five years, you don't begin with how inspiring the founder is. You begin with the survival rate of startups in general, then adjust up or down for what's genuinely special about this one.
The tournament's own data points the same way. In its final year, trained forecasters could tick which of ten principles an explanation relied on, and only one was significantly linked to better accuracy: comparison classes, the outside view by another name.
Getting the order right matters. Anchor on the vivid inside story first and every adjustment is contaminated by it. Anchor on the base rate first and the details become corrections to a sensible starting point instead of the whole basis for your guess. This is the same fox-like instinct, breadth before depth, that how to apply Range argues makes generalists so effective in unpredictable domains.
Think in Probabilities, Not Words
Ordinary language is where forecasts go to die. In March 1951, a U.S. National Intelligence Estimate concluded that a Soviet-bloc attack on Yugoslavia "should be considered a serious possibility." When Sherman Kent, a CIA analyst who had helped produce it, asked colleagues on the Board of National Estimates what odds they'd had in mind, their readings ran from 20 percent to 80 percent. Kent's own was about 65 percent. Everyone had agreed on the sentence, and almost nobody agreed on what it meant. A forecast like that can't be scored, so it can never be wrong, so you can never learn from it.
Superforecasters refuse to hide there. They translate hunches into specific probabilities and they use fine gradations, 63 percent rather than a round "likely." That precision isn't false confidence, it's the opposite. When Barbara Mellers and her colleagues rounded superforecasters' estimates to the nearest 0.1, their accuracy dropped, which means those small distinctions carried real information. The tool for keeping score is the Brier score, which measures both calibration (do things you call 70 percent happen about 70 percent of the time?) and resolution (do you take decisive stands instead of hiding at 50 percent?). Lower is better.
| Forecast style | What it sounds like | Can you learn from it? |
|---|---|---|
| Vague words | "Probably," "a serious possibility," "unlikely" | No, it can't be scored |
| Round numbers | "About 70 percent" | A little, but coarse |
| Fine-grained probabilities | "63 percent, and here's why" | Yes, calibration improves over time |
Numbers do something words can't: they turn a prediction into feedback. Say "70 percent" a hundred times and you can check whether roughly 70 of those things happened. Say "probably" a hundred times and you've learned nothing about yourself. Numbers also guard against what Tetlock calls the wrong-side-of-maybe fallacy, judging a 70 percent forecast a failure because the 30 percent came in. One miss tells you almost nothing. A hundred forecasts tell you a lot.
Update Your Beliefs Like a Bayesian
A forecast made once and defended forever is just an opinion. The heart of the method is updating: adjusting your probability as new evidence arrives, in proportion to how much that evidence tells you. This is Bayesian reasoning in spirit, and superforecasters do it constantly, in small increments, without drama.
The discipline cuts two ways. Underreaction is clinging to a prior forecast because you're anchored, invested, or embarrassed to move. Overreaction is lurching to a new number every time a fresh headline lands, treating noise as signal. The skill is calibrating the size of the update to the weight of the evidence: a minor news item nudges you a few points, a genuine surprise moves you a lot. Superforecasters were markedly better at both, changing their minds more often than average forecasters but also more precisely.
What makes this hard isn't math, it's ego. Updating means publicly conceding your last forecast was less than perfect, and most people would rather be consistent than accurate. The fix is to stop treating a changed mind as a defeat. Annie Duke frames the same move in Thinking in Bets as separating your beliefs from your identity: the question is never "was I right," it's "what do I believe now, given everything I know today."
Build a Forecasting Practice You Actually Keep
Everything above collapses without a record. If you don't write down what you predicted, when, and why, memory quietly rewrites the story so you were always basically right. The single most practical thing you can take from Superforecasting is a forecasting journal, and Tetlock's Ten Commandments are the checklist that goes in it.
| Commandment | What it means in practice |
|---|---|
| Triage | Spend effort on questions where work actually pays off, not the trivial or the hopeless |
| Break problems down | Fermi-ize: split a big question into answerable pieces |
| Balance outside and inside views | Start from base rates, then adjust for specifics |
| Update your beliefs | Revise in small, evidence-weighted steps |
| Look for clashing causal forces | Hold competing explanations in mind at once |
| Use the right degree of doubt | Translate hunches into precise probabilities |
| Balance prudence and decisiveness | Don't rush to certainty or hide at "maybe" |
| Learn from failure and success | Run unflinching postmortems on both |
| Master perspective-taking in teams | Disagree productively, ask precise questions |
| Practice deliberately with feedback | Treat forecasting like a trainable craft |
Tetlock adds an eleventh, half-joking commandment: don't treat the commandments as commandments. They're guidelines for a messy world, not laws.
This is where your reading tools become forecasting tools. When you read an analysis or watch a briefing and it makes a claim about the future, capture it. With Glasp's web highlighter you can highlight the exact prediction and add your own probability as a note, right on the source. For long video arguments, YouTube video summaries let you pull out the key claims and timestamp them so you can log a forecast against each one.
If you're reading Superforecasting itself on a Kindle, import your Kindle highlights so the commandments you marked sit next to the forecasts you make. Over months those annotations become a searchable track record, and Glasp's AI chat lets you ask questions across everything you've saved when you review your own reasoning. It's the reading-side version of keeping score, and it pairs naturally with the digital commonplace book approach to collecting what you learn.
How to Become a Superforecaster
There's no exam and no course that hands you the title. You earn it the way the original superforecasters did, with a long record of scored forecasts that beats almost everyone else forecasting the same questions. A few lucky calls prove nothing. Hundreds of scored ones start to.
Three habits matter most:
- Forecast where you're scored. Good Judgment Open is the public forecasting site run by Good Judgment Inc., open to anyone, and Metaculus is free to join and runs seasonal tournaments such as the Metaculus Cup. Both score you against reality and against everyone else on the same question.
- Answer many questions, and keep updating your forecasts. Accuracy needs a sample. Superforecasters were picked on a whole season's record, and they revised their forecasts as news arrived instead of making one call and walking away.
- Write down your reasoning. Good Judgment says it recruits potential Superforecasters from Good Judgment Open each autumn and asks hopefuls to forecast at least 100 questions there. It looks closest at average accuracy, but also at the quality of written comments and at collegiality. Those it picks may get a trial engagement, and recruits must complete a three-month probation. Metaculus picks its Pro Forecasters for their track record and for "their history of clearly describing their rationales."
| Where to practice | What you get | Path to a title |
|---|---|---|
| Good Judgment Open | Scored questions on world events, leaderboards | Consistent top performers are eligible for Good Judgment's annual professional Superforecaster selection |
| Metaculus | Questions, tournaments, community forecasts, scores | Pro Forecasters are chosen for track record and clearly explained rationales |
| A personal forecasting journal | Questions from your own work and reading | None, but it's where calibration starts |
Don't skip the journal because the platforms exist. Platform questions are written by someone else. The forecasts that sharpen your judgment most are about questions you pick yourself: whether a project ships on time, whether a source's prediction holds up, whether the trend in the article you just read continues.
Superforecasters vs. AI: Where Things Stand in 2026
On clear questions that resolve within months, the best AI forecasting systems have caught up with superforecasters, and on questions generated from data series they have probably pulled ahead. On one-off questions that call for judgment, nobody has shown a statistically clear lead either way. The team that built the main benchmark and Good Judgment Inc., which runs the professional Superforecaster panel, read the same results very differently.
ForecastBench, built by the Forecasting Research Institute (FRI), scores AI systems against superforecasters on two kinds of questions: dataset questions generated from data series such as economic indicators, and market questions drawn from prediction platforms. In January 2026, superforecasters still ranked first, ahead of the best models by 0.017 Brier points, which FRI estimated at about a year of model progress. It projected parity for November 2026.
It came sooner. On July 16, 2026, FRI reported that several systems, led by one from Cassi AI and including entries from xAI and Google DeepMind, were statistically indistinguishable from superforecasters on its tournament leaderboard. FRI said Cassi's system was also the first to rank above the superforecaster median on market questions. AI had matched superforecasters on dataset questions back in May, and by July FRI said many systems likely outperform them there. Its caveats were candid: it last collected superforecaster forecasts in 2024, so the comparison rests on a statistical adjustment across question sets with no questions in common, and the confidence intervals overlap so much that the results "are more consistent with superforecaster parity than with outperformance."
Good Judgment pushed back eight days later in a post titled "Not So Fast," with three objections:
- The human baseline is stale. It called it "a frozen July 2024 snapshot of individual Superforecasters working largely alone, with minimal updating," when the Good Judgment Project's own experiments showed teams beating individuals.
- Some market questions are data questions in disguise. Questions like whether July 2026 would be the warmest July on record are, in its words, "dataset questions in market clothing," resolvable from climate records.
- Models may be copying the market. FRI had earlier found that one model's forecasts correlated 0.994 with the market prices it was shown, which raises the question of how much of an AI's accuracy is the market's.
Metaculus runs the other major comparison, FutureEval, which pits the ten best bots from its bot tournament against ten of its Pro Forecasters on the same questions. In the spring 2026 round, the Pros won by 1.25 head-to-head points per question over 99 questions, a margin Metaculus called not statistically significant. In the first two quarters of 2025, the gap had been 17.7 and 20.0 points. Humans still took nine of the top ten places when everyone was ranked individually. Then came the summer 2026 Metaculus Cup, where bots forecast the same questions as human contestants but can't win prizes. Two bots, Laertes and Mantic's system, finished first and second overall, ahead of every human contestant.
| Benchmark | Who is compared | Latest result |
|---|---|---|
| ForecastBench (FRI) | AI systems vs. a 2024 superforecaster baseline | July 2026: several systems statistically indistinguishable from superforecasters, and Cassi's system ranked above them on market questions |
| Metaculus FutureEval | Top-10 bot team vs. 10 Pro Forecasters | Spring 2026: Pros ahead by 1.25 points per question, not statistically significant |
| Metaculus Cup | Bots on the same questions as human contestants | Summer 2026: bots finished first and second overall |
For a reader, two things follow. First, the people this book describes became the bar AI forecasting is measured against, and the best models only reached it in 2026. Second, an AI forecast is one more source to weigh. Treat it the way a superforecaster treats a pundit: ask for the base rate, check whether it's just echoing the market, and write its claim down so you can score it later.
The Honest Limits of Superforecasting
Superforecasting isn't magic, and Tetlock is careful about its boundaries. The tournament questions had clear resolution dates and definite answers, and most looked between one month and one year ahead. That's exactly the zone where the method shines and exactly where most of life's biggest questions don't live. Ask "will there be a war in the next decade" and there's no clean way to score it, no rapid feedback, and no way to practice.
The sharpest critique comes from Nassim Taleb, who argues that the events that truly reshape the world are the rare, high-impact "black swans" that these tournaments systematically exclude. Being well-calibrated on hundreds of ordinary questions, he'd say, tells you little about the one tail event that matters. Tetlock's reasonable reply is that short-run forecasting skill and long-run humility can coexist: getting the knowable questions right is worth doing even if the unknowable ones remain unknowable.
Even on questions that do resolve, the ground can shift under the method. In FRI's Existential Risk Persuasion Tournament, held in 2022, superforecasters gave just a 2.3 percent chance that an AI system would reach gold-medal level at the International Mathematical Olympiad by 2025. It happened in July 2025. Across four AI benchmarks whose questions have now resolved, superforecasters had given the actual outcomes an average probability of 9.7 percent, against 24.6 percent from domain experts. Base rates come from the past, and they mislead fastest when the process producing them is changing.
There's a subtler trap too. The scoring can pull you toward safe, near-term questions where you'll look good, and away from the vague, important ones where judgment matters most. The method is a tool for a specific kind of question, not a worldview. Used well, it sharpens your thinking. Used badly, it becomes a way to feel precise about things that don't count.
Frequently Asked Questions
What is a superforecaster?
A superforecaster is a person whose probability forecasts have been shown, over many scored questions, to be consistently more accurate than those of almost everyone else forecasting the same questions. The term comes from Philip Tetlock's Good Judgment Project, which each year gave the title to its top 2 percent of forecasters. About 70 percent kept the title the following year. Good Judgment Inc. still uses it for its professional forecasting panel.
Who are the most famous superforecasters?
Two of the best known are profiled in Tetlock's book Superforecasting. Doug Lorch, a retired computer programmer, finished fifth of about 2,800 forecasters in his first year and posted the best score in the project the next. Bill Flack, a former U.S. Department of Agriculture employee from Nebraska, was another standout. Neither was a professional forecaster or intelligence analyst.
Are superforecasters better than AI?
Not clearly, as of 2026. In July 2026 the Forecasting Research Institute reported that several AI systems were statistically indistinguishable from superforecasters on ForecastBench, and that many likely do better on questions generated from data series, though its human baseline dates from 2024. Metaculus Pro Forecasters still led the best bot team in spring 2026, by a margin too small to be statistically significant, but in the summer 2026 Metaculus Cup two bots finished ahead of every human contestant. Good Judgment says that in its own client work, a small sample, it hasn't found AI to equal superforecasters.
How do you become a superforecaster?
Build a long, scored record. Forecast regularly on Good Judgment Open or Metaculus, answer many questions rather than a few, update as news arrives, and write down your reasoning. Good Judgment says it recruits potential Superforecasters from Good Judgment Open each autumn, asks hopefuls to forecast at least 100 questions there, looks at accuracy, comment quality and collegiality, may offer a trial engagement, and makes recruits complete a three-month probation.
Can superforecasting be learned?
Yes, and that's the book's central finding. Superforecasters scored above average on intelligence but weren't geniuses, and Tetlock writes that the strongest predictor of becoming one was "perpetual beta," a commitment to updating your beliefs and improving yourself, roughly three times as powerful as intelligence. In the tournament's randomized training experiment, about an hour of training improved ordinary forecasters' accuracy by 6 to 11 percent.
How is Superforecasting different from Thinking in Bets?
They're complementary. Annie Duke's Thinking in Bets focuses on decision quality under uncertainty and on separating good decisions from lucky outcomes. Tetlock's Superforecasting focuses on the prediction itself: how to assign accurate probabilities to future events and measure whether you were calibrated. One is about deciding well, the other about seeing clearly. Read together they cover both halves of reasoning under uncertainty.
What is a Brier score in simple terms?
It's a way to grade probabilistic forecasts, where lower is better and zero is perfect. It rewards two things at once: calibration (things you call 70 percent should happen about 70 percent of the time) and resolution (taking decisive stands instead of always hedging near 50 percent). You need a running log of predictions and outcomes to compute it, which is why keeping a forecasting journal is the foundation of the whole practice.
Do I need math to become a better forecaster?
No advanced math is required. The core moves are habits: break a question into parts, start from a base rate, express your view as a specific number, and update in small steps as evidence arrives. Bayesian updating is a mindset before it's an equation. The real bottleneck isn't calculation, it's the willingness to write down your predictions, check them honestly, and change your mind.
How do I start practicing today?
Pick a handful of questions with a clear answer and a near deadline, from your work, your reading, or the news. Write down a specific probability for each and one sentence on your reasoning. When each resolves, compare and run a short postmortem. Highlighting predictions as you read, with a probability attached in the margin, turns your normal reading into a steady stream of practice questions.
Conclusion
The quiet radicalism of Superforecasting is its optimism. For decades the takeaway from Tetlock's work was cynical: experts can't predict, so why bother. The tournament flipped it. A group of ordinary people, armed with a method and the discipline to keep score, learned to see the future more clearly than professionals with secret files. The method is teachable, and most of it comes down to habits any careful reader can build. Now that the best AI systems have drawn level with superforecasters on the main benchmarks, the same habits are how you judge any forecast you're handed, whether a person or a model made it.
Start small. Break questions apart, anchor on how often things like this usually happen, put a real number on your belief, and update it without ego as the world talks back. Then write it down. Use Glasp's web highlighter to capture the claims you meet while reading and note your own probability on each, lean on YouTube video summaries to log forecasts against the arguments you watch, and let Glasp's community feed show you how other careful readers reason through the same uncertainty. Forecasting well is a skill. Like every skill worth having, it grows from deliberate practice, honest feedback, and a record you can learn from. For the note-taking system that holds it all together, see how to take smart notes.
