How to Apply Superforecasting: Train Your Judgment to Predict Better

The Book That Made Forecasting a Measurable Skill

In 2011 the Intelligence Advanced Research Projects Activity, a research arm of the U.S. intelligence community, ran a strange contest. It wanted to know whether anyone could actually forecast world events, so it invited university teams to compete. Each team recruited volunteers to answer hundreds of hard geopolitical questions with numeric probabilities: Will this country default? Will these two nations fight? Will this leader still hold power in six months?

Philip Tetlock and Barbara Mellers ran one of those teams, the Good Judgment Project. Over four years, thousands of online volunteers made more than a million forecasts across roughly 500 questions. The results embarrassed the professionals. Good Judgment beat the other university teams by margins of 35 to 72 percent on accuracy, and its best forecasters were about 30 percent more accurate than intelligence analysts who had access to classified information the volunteers never saw.

The book Tetlock wrote with journalist Dan Gardner, Superforecasting: The Art and Science of Prediction (2015), is the story of those people and what made them good. One of them, a retired computer programmer named Doug Lorch, made roughly a thousand forecasts in the first year alone and finished fifth out of about 2,800 competitors. He wasn't a spy or a quant. He was a hobbyist who'd found a system that worked.

Most people read Superforecasting to get better at predicting markets or politics. This guide treats it as something more useful to a reader: a training manual for judgment. The skill Tetlock isolates, thinking clearly about an uncertain future, is the same skill you use every time you decide what to believe, which sources to trust, and what to do with what you learn.

Why Most Expert Predictions Fail

Tetlock earned the right to write this book the hard way. His earlier work, Expert Political Judgment (2005), tracked hundreds of professional pundits and analysts making tens of thousands of predictions over about twenty years. The famous verdict: the average expert was roughly as accurate as "a dart-throwing chimpanzee." Worse, the more famous the expert, the worse the calibration tended to be, because confident, sweeping stories get you on television while careful hedging does not.

But the averages hid a split. Borrowing Isaiah Berlin's metaphor, Tetlock sorted forecasters into hedgehogs and foxes. The hedgehog knows one big thing and forces every question through a single grand theory. The fox knows many small things, distrusts big theories, and stitches together evidence from lots of sources. Foxes consistently forecast better. Hedgehogs made bolder predictions and got more attention, but their one big idea kept steering them into confident errors.

The lesson isn't that expertise is useless. It's that being knowledgeable and being well-calibrated are different skills. Knowing a lot about a topic tells you what could happen. It doesn't automatically tell you how likely each outcome is, and likelihood is what a forecast is actually about. That gap is why smart, informed people are so often confidently wrong, a pattern that Daniel Kahneman spent a career documenting in how to apply Thinking, Fast and Slow.

What Actually Makes a Superforecaster

The obvious guess is intelligence. Superforecasters did score higher than average on tests of intelligence and knowledge, roughly in the top 20 percent, but they weren't outliers. Plenty of very smart people forecast poorly, and plenty of merely bright ones excelled. IQ set a floor, not a ceiling.

What separated the best was a habit of mind Tetlock calls "perpetual beta," borrowing the software term for a product that's never finished and always being improved. Superforecasters treat every belief as a draft. They actively hunt for reasons they might be wrong, they revise in small steps as evidence trickles in, and they run honest postmortems on their misses. Tetlock found that this commitment to updating and self-improvement was about three times more powerful a predictor of accuracy than intelligence itself.

The people themselves were unremarkable on paper. Bill Flack, one of the standout forecasters, was a retired U.S. Department of Agriculture employee from Nebraska who liked bird-watching. Others came to it after a painful miss: some joined the project precisely because they'd failed to see something coming, like the 2008 financial crisis, and wanted to get better. That's the growth mindset that Carol Dweck describes in how to apply Mindset, pointed at a single trainable skill. The reassuring implication of the whole book is that superforecasting is learned behavior, not a personality you're stuck without.

Fermi-ize the Question and Start From the Outside

Faced with a big, vague question, most people reach for their gut and a headline. Superforecasters do the opposite. They break the question into smaller sub-questions, each easier to reason about, a technique Tetlock calls "Fermi-izing" after the physicist Enrico Fermi, who was famous for estimating hard quantities (like the number of piano tuners in Chicago) by decomposing them. The point of breaking a problem apart is to separate what you actually know from what you're guessing, or as Tetlock puts it, to "flush ignorance into the open."

The next move is counterintuitive: start from the outside, not the inside. The inside view is the specific story in front of you, all its vivid details. The outside view asks a colder question: how often do things like this happen in situations like this? That base rate is your anchor. If you want to forecast whether a startup will still exist in five years, you don't begin with how inspiring the founder is. You begin with the survival rate of startups in general, then adjust up or down for what's genuinely special about this one.

Getting the order right matters. Anchor on the vivid inside story first and every adjustment is contaminated by it. Anchor on the base rate first and the details become corrections to a sensible starting point instead of the whole basis for your guess. This is the same fox-like instinct, breadth before depth, that how to apply Range argues makes generalists so effective in unpredictable domains.

Think in Probabilities, Not Words

Ordinary language is where forecasts go to die. "There's a real chance of a recession" feels like a claim, but it can't be scored, so it can never be wrong, so you can never learn from it. Studies of vague-probability words show the problem plainly: when people are asked what "a real possibility" means numerically, answers range from about 20 percent to over 80 percent. Everyone hears their own number.

Superforecasters refuse to hide there. They translate hunches into specific probabilities and they use fine gradations, 63 percent rather than a round "likely." That precision isn't false confidence, it's the opposite. Tetlock found that when researchers rounded superforecasters' estimates to coarser numbers, their accuracy dropped, which means those small distinctions carried real information. The tool for keeping score is the Brier score, which measures both calibration (do things you call 70 percent actually happen about 70 percent of the time?) and resolution (do you take decisive stands instead of hiding at 50 percent?). Lower is better.

Numbers do something words can't: they turn a prediction into feedback. Say "70 percent" a hundred times and you can check whether roughly 70 of those things happened. Say "probably" a hundred times and you've learned nothing about yourself.

Update Your Beliefs Like a Bayesian

A forecast made once and defended forever is just an opinion. The heart of the method is updating: adjusting your probability as new evidence arrives, in proportion to how much that evidence actually tells you. This is Bayesian reasoning in spirit, and superforecasters do it constantly, in small increments, without drama.

The discipline cuts two ways. Underreaction is clinging to a prior forecast because you're anchored, invested, or embarrassed to move. Overreaction is lurching to a new number every time a fresh headline lands, treating noise as signal. The skill is calibrating the size of the update to the weight of the evidence: a minor news item nudges you a few points, a genuine surprise moves you a lot. Superforecasters were markedly better at both, changing their minds more often than average forecasters but also more precisely.

What makes this hard isn't math, it's ego. Updating means publicly conceding your last forecast was less than perfect, and most people would rather be consistent than accurate. The fix is to stop treating a changed mind as a defeat. Annie Duke frames the same move as separating your beliefs from your identity in how to apply Thinking in Bets: the question is never "was I right," it's "what do I believe now, given everything I know today."

Build a Forecasting Practice You Actually Keep

Everything above collapses without a record. If you don't write down what you predicted, when, and why, memory quietly rewrites the story so you were always basically right. The single most practical thing you can take from Superforecasting is a forecasting journal, and Tetlock's Ten Commandments are the checklist that goes in it.

Tetlock adds an eleventh, half-joking commandment: don't treat the commandments as commandments. They're guidelines for a messy world, not laws.

Here's where your reading tools become forecasting tools. When you read an analysis or watch a briefing and it makes a claim about the future, capture it. With Glasp's web highlighter you can highlight the exact prediction and add your own probability as a note, right on the source. For long video arguments, YouTube video summaries let you pull out the key claims and timestamp them so you can log a forecast against each one. Over months those annotations become a searchable track record, and Glasp's AI chat can help you review your own reasoning by asking questions across everything you've saved. The habit of logging a probability every time you meet a claim is the reading-side version of keeping score, and it pairs naturally with the digital commonplace book approach to collecting what you learn.

The Honest Limits of Superforecasting

Superforecasting isn't magic, and Tetlock is careful about its boundaries. The tournament questions had clear resolution dates and definite answers, usually within a year or two. That's exactly the zone where the method shines and exactly where most of life's biggest questions don't live. Ask "will there be a war in the next decade" and there's no clean way to score it, no rapid feedback, and no way to practice.

The sharpest critique comes from Nassim Taleb, who argues that the events that truly reshape the world are the rare, high-impact "black swans" that these tournaments systematically exclude. Being well-calibrated on hundreds of ordinary questions, he'd say, tells you little about the one tail event that actually matters. Tetlock's reasonable reply is that short-run forecasting skill and long-run humility can coexist: getting the knowable questions right is worth doing even if the unknowable ones remain unknowable.

There's a subtler trap too. The scoring can pull you toward safe, near-term questions where you'll look good, and away from the vague, important ones where judgment matters most. The method is a tool for a specific kind of question, not a worldview. Used well, it sharpens your thinking. Used badly, it becomes a way to feel precise about things that don't count.

Frequently Asked Questions

Is superforecasting something you're born with or can learn?

Learn. That's the book's central finding. Superforecasters scored above average on intelligence but weren't geniuses, and the strongest predictor of skill wasn't IQ but "perpetual beta," the habit of constantly updating and improving your beliefs. Tetlock found it about three times more predictive of accuracy than intelligence. In the tournament's training experiments, even a short tutorial on techniques like base rates measurably improved ordinary forecasters.

How is Superforecasting different from Thinking in Bets?

They're complementary. Annie Duke's Thinking in Bets focuses on decision quality under uncertainty and on separating good decisions from lucky outcomes. Tetlock's Superforecasting focuses on the prediction itself: how to assign accurate probabilities to future events and measure whether you were calibrated. One is about deciding well, the other about seeing clearly. Read together they cover both halves of reasoning under uncertainty.

What is a Brier score in simple terms?

It's a way to grade probabilistic forecasts, where lower is better and zero is perfect. It rewards two things at once: calibration (things you call 70 percent should happen about 70 percent of the time) and resolution (taking decisive stands instead of always hedging near 50 percent). You need a running log of predictions and outcomes to compute it, which is why keeping a forecasting journal is the foundation of the whole practice.

Do I need math to become a better forecaster?

No advanced math is required. The core moves are habits: break a question into parts, start from a base rate, express your view as a specific number, and update in small steps as evidence arrives. Bayesian updating is a mindset before it's an equation. The real bottleneck isn't calculation, it's the willingness to write down your predictions, check them honestly, and change your mind.

How do I start practicing today?

Pick a handful of questions with a clear answer and a near deadline, from your work, your reading, or the news. Write down a specific probability for each and one sentence on your reasoning. When each resolves, compare and run a short postmortem. Highlighting predictions as you read, with a probability attached in the margin, turns your normal reading into a steady stream of practice questions.

Conclusion

The quiet radicalism of Superforecasting is its optimism. For decades the takeaway from Tetlock's work was cynical: experts can't predict, so why bother. The tournament flipped it. A group of ordinary people, armed with a method and the discipline to keep score, learned to see the future more clearly than professionals with secret files. The method is teachable, and most of it comes down to habits any careful reader can build.

Start small. Break questions apart, anchor on how often things like this usually happen, put a real number on your belief, and update it without ego as the world talks back. Then, crucially, write it down. Use Glasp's web highlighter to capture the claims you meet while reading and tag each with your own probability, lean on YouTube video summaries to log forecasts against the arguments you watch, and let Glasp's community feed show you how other careful readers reason through the same uncertainty. Forecasting well is a skill. Like every skill worth having, it grows from deliberate practice, honest feedback, and a record you can actually learn from. For the note-taking system that holds it all together, see how to take smart notes.