Why the Best Ways to Prove Impact Start by Admitting You Cannot Prove It Cleanly

The uncomfortable truth about impact

What if the most sophisticated way to understand change is not to hunt for the single cause that “worked,” but to accept that real-world change is almost always a messy chain of partial influences? That question sits at the heart of a tension most organizations try to avoid. We want proof, but the world gives us complexity. We want a neat causal story, but social change arrives through feedback loops, competing forces, and outcomes that emerge long after the original intervention has faded from view.

This is why impact evaluation so often feels trapped between two unsatisfying extremes. On one side is the demand for certainty: if we cannot isolate the effect of one program, the thinking goes, perhaps we should trust only what is directly measurable. On the other side is the reality that many important changes cannot be reduced to a single variable. In practice, people, institutions, and communities change through overlapping pathways. The challenge is not whether causality exists, but how to trace it responsibly when the world refuses to behave like a laboratory.

The deepest insight here is simple but disruptive: accountability does not require illusionary precision. It requires a disciplined way of building and testing a causal story from the evidence that actually exists. Once you see that, evaluation stops being a courtroom demanding a single guilty party and becomes more like detective work, assembling clues into the most plausible explanation of what changed, why it changed, and what role a particular effort played.

---

The false comfort of single-cause thinking

Many organizations still think about results as if they were produced by a chain with one dominant link. We funded this project, therefore this outcome happened. But real change is more like weather than machinery. A rainfall pattern may depend on evaporation, temperature, wind, geography, and seasonality at once. Try assigning the storm to one cause and you miss the system. Social outcomes behave similarly: a literacy program may matter, but so may teacher morale, household income, language policy, nutrition, and access to books.

This is where causal pathways become more useful than blunt attribution. A causal pathway asks: what sequence of conditions, interactions, and intermediate steps plausibly connects an action to an observed result? Instead of pretending there is one lever, it treats change as a route. The route may be interrupted, accelerated, reshaped, or amplified by factors outside the intervention. Yet the pathway can still be traced with care.

Consider a health campaign aimed at increasing vaccine uptake. If uptake rises, did the posters do it? The clinic hours? A trusted local leader? A social media rumor that backfired? A causal pathway lens would not stop at the final number. It would ask which intermediate changes occurred: awareness, intention, access, trust, peer pressure, and behavior. That is much closer to how people actually decide.

The question is not “What caused the outcome?” but “What chain of changes made the outcome possible, and where did this effort fit inside that chain?”

This shift matters because it changes what counts as evidence. A single outcome may be overdetermined, meaning many causes contributed. If so, the right task is not to isolate a pure effect but to identify the contribution structure of the change. That is a more honest ambition, and often a more useful one.

---

From attribution to contribution: a better standard of proof

There is a subtle but powerful difference between saying, “We caused this outcome,” and saying, “We contributed credibly to this outcome.” The first implies ownership of the result. The second implies a disciplined argument that the initiative was part of a real causal story. In complex environments, contribution is often the highest defensible standard.

A contribution claim asks four questions:

1. Was the outcome plausible given the intervention and context?
2. Did intermediate changes appear in the expected sequence?
3. Is there evidence that the intervention influenced those changes?
4. Are alternative explanations weaker, incomplete, or less consistent with the timeline?

This is not about lowering standards. It is about matching the standard to the nature of the phenomenon. If you are trying to assess whether a policy shifted behavior across a district, you may not be able to run a perfect experiment. But you can still examine timing, mechanisms, comparison cases, stakeholder testimony, administrative records, and observable intermediate effects. The result is not mathematical certainty. It is a well-supported causal narrative.

That narrative becomes especially powerful when it is explicit about uncertainty. Most reporting hides uncertainty behind polished dashboards. Yet uncertainty is not a weakness if it is structured. A causal pathway approach makes the unknown visible: which links are strong, which are inferred, which are contested, and which remain unobserved. This transparency often builds more trust than overconfident numbers do.

The practical advantage is enormous. Teams can learn not only whether something worked, but where it worked, for whom, under what conditions, and through which mechanisms. That turns evaluation from a binary verdict into a diagnostic tool.

---

Why outcomes matter more than activities, and less than you think

A second temptation is to evaluate organizations by their activities: number of workshops held, brochures printed, people trained, meetings convened. These are easy to count, but they are not change. They are inputs or outputs, not outcomes. The danger is that activity metrics create the illusion of momentum while leaving the real transformation untouched.

Outcome-focused thinking corrects that error, but it can go too far in the opposite direction. If we only track final outcomes, we may miss the living mechanisms that make those outcomes possible. A farmers’ training program might not immediately raise yields, but it may alter planting decisions, reduce input waste, improve experimentation, or increase peer learning. Those intermediate outcomes are not trivial. They are the pathway itself.

This is why outcomes should be treated as evidence of motion, not just the destination. An organization that wants durable change needs to monitor the stages between effort and result. Think of it like navigating with landmarks instead of only checking whether you reached the city. If you see the right bridges, roads, and signs, you can tell whether you are on course long before arrival.

A concrete example makes this clearer. Imagine a youth employment initiative that provides job search coaching, employer connections, and interview practice. The final outcome is employment. But the pathway may include improved confidence, stronger resumes, more interviews, and better employer perceptions. If employment does not rise, the intermediate outcomes reveal where the mechanism broke. Maybe confidence improved but employers were not hiring. Maybe interviews increased but wages were too low. Maybe participants gained skills but lacked transportation.

That is the real value of a pathway approach. It identifies which link in the chain deserves attention. Without that, an evaluation can tell you that something failed, but not what to fix.

---

Outcome harvesting: when change appears before the plan does

Not all valuable change comes from a neatly designed plan. Sometimes the most important shifts are unexpected, emergent, or politically messy. A new relationship forms, a local actor adopts a practice, a policy debate changes tone, a partner uses a tool in an unanticipated way. In such settings, rigid evaluation frameworks can miss the story entirely because they look only for intended results.

A more responsive approach is to begin with the outcomes that actually occurred and then work backward to understand what influenced them. This matters especially in complex systems where interventions interact with context in unpredictable ways. A project might intend to improve access to clean water, yet end up strengthening community governance, shifting gender roles, or reshaping local trust in public institutions. Those may be just as significant as the original target.

Here is the deeper insight: outcomes are not only endpoints, they are clues. They signal where change has surfaced, where it has traveled, and what ripples an initiative may have created. If you ignore unexpected outcomes, you risk evaluating only your plans, not your impact.

This is where harvesting becomes a mindset, not just a method. It means scanning for evidence of change wherever it appears, then asking how it came about. A field team hears that a local school adopted a new attendance practice after an informal conversation, not after the official training. That is not noise. It may reveal an indirect pathway that is more important than the formal one. The real influence of a program often lies in these side doors, not the front door.

If causal pathway analysis asks, “How did this happen?”, outcome harvesting asks, “What changed that matters, and what does that reveal about the system?”

Together, they create a more complete picture. One traces mechanism. The other detects emergence. One helps explain planned change. The other helps reveal unplanned but meaningful change. In complex environments, you need both.

---

A new mental model: the causal thread and the outcome net

The best synthesis of these ideas is to think in terms of two complementary structures: a causal thread and an outcome net.

The causal thread is the pathway you expect to connect action to result. It is directional, sequenced, and mechanism focused. It answers questions like: What needs to happen first? Which intermediate changes are necessary? Where might the chain break?

The outcome net is the broader field of actual changes that appear around the intervention. It is exploratory, adaptive, and evidence gathering. It answers questions like: What shifted, intended or not? Which outcomes are linked by timing, relationships, or shared causes? Which changes are early signals of a deeper transformation?

Think of a public health initiative using both lenses. The causal thread might run from outreach to awareness, from awareness to clinic visits, from clinic visits to treatment adherence, and from adherence to better health. The outcome net might reveal unexpected changes such as increased trust in local health workers, new peer support groups, or policy adjustments at the municipal level. Some of these may be side effects. Others may be the real story.

This dual model is valuable because it resists two common mistakes. The first mistake is tunnel vision, where only the planned pathway matters and surprise is ignored. The second is scatter, where everything interesting is treated as impact without a causal structure. The causal thread gives discipline. The outcome net gives openness. Together they make evaluation both rigorous and alive.

A good evaluator, then, is not merely a scorekeeper. They are a systems reader. They look for pattern, sequence, substitution, spillover, and resonance. They ask not only whether change happened, but how the pieces fit together into a believable account.

---

Key Takeaways

1. Stop demanding a single cause when change is multi-causal. In complex settings, the goal is credible contribution, not false certainty.

2. Track intermediate outcomes as seriously as final outcomes. Intermediate shifts reveal whether the mechanism is working, stalled, or being redirected.

3. Treat unexpected outcomes as evidence, not distraction. Surprising changes may expose the real influence of an intervention.

4. Build causal narratives with multiple forms of evidence. Use timing, context, stakeholder accounts, records, and comparison cases to test plausibility.

5. Use two lenses at once: the causal thread and the outcome net. One keeps your theory disciplined. The other keeps your perception open.

---

The real payoff: humility that improves action

The deepest value of these approaches is not methodological sophistication for its own sake. It is epistemic humility with operational consequences. When organizations admit that change is partial, indirect, and context dependent, they become better at learning. They stop overclaiming. They stop mistaking activity for impact. They start asking more precise questions about what actually moved the system.

That humility is not passive. It is one of the most practical forms of intelligence. If you know which links in the causal chain are strong, you can reinforce them. If you know where outcomes are emerging unexpectedly, you can decide whether to nurture them, study them, or replicate them elsewhere. If you know which parts of your story are weak, you can test them instead of hiding them.

So the next time someone asks whether a program “worked,” consider refusing the false simplicity of the question. A better question is: What changed, through what pathway, with what contribution from this effort, and what does that teach us about the system? That question is harder. It is also more truthful, more useful, and ultimately more respectful of the complexity of real life.

The future of evaluation belongs to those who can hold both rigor and ambiguity at once. Not because they have abandoned proof, but because they have learned that in the real world, proof often looks like a carefully assembled story about how change happened, not a single number that pretends to end the conversation.