The Hidden Variable Problem: Why Better Explanations Start by Reconstructing What You Never Saw

Jun 06, 2026

10 min read

4 views

72%

The real question is not what caused the outcome, but what was missing from the story

We usually treat prediction and explanation as two separate tasks. First, a model forecasts an outcome. Then, an explanation layer tells us which features mattered. But this sequence hides a deeper problem: what if the most important information was never directly observed in the first place?

That is the tension connecting modern causal inference and model interpretation. In many real systems, the data we see are fragments, not wholes. A patient’s health trajectory is interrupted by missed visits. A company’s performance is shaped by unrecorded strategy shifts. A region’s economic output is affected by shocks that are only partially captured. In such settings, the question is not merely, “Which feature drove the prediction?” It is, “What latent structure would make the observed pattern coherent?”

This is where two ideas meet in a surprisingly powerful way. Matrix completion asks whether a partially observed world can be reconstructed from patterns in the visible entries. SHAP-style interpretation asks how much each feature contributed to a prediction, conditional on the values of that feature and the model. One works by filling in the blanks. The other works by assigning credit. Together, they suggest a new mental model: good explanation often depends on first rebuilding the counterfactual world the data failed to record.

We do not understand an outcome until we understand the missing structure that made the outcome appear inevitable.

Why prediction without reconstruction can be a form of illusion

Most machine learning workflows quietly assume that the observed data are complete enough to support meaning. Yet in causal panel settings, this assumption is often false. You may observe a unit over time, but not all units at all times. You may see outcomes after treatment, but not the untreated path. You may have covariates that are measured inconsistently, or hidden confounders that leave their trace only indirectly.

A matrix completion perspective starts from a humbler premise: the data are a damaged object. The task is not to invent values arbitrarily, but to exploit low-dimensional regularities so the missing pieces become inferable from the observed ones. Think of a mural with several tiles missing. If the surrounding pattern has structure, you can often reconstruct the absent tiles with remarkable accuracy. The key is not magic, but geometry. The world may be high-dimensional on the surface, but the relevant dynamics often live on a much smaller manifold.

This has an immediate implication for interpretation. If a model is trained on incomplete or distorted data, then the explanation of its predictions can be misleading unless it is anchored in a reconstructed baseline. A feature may appear influential simply because it is proxying for something unobserved. Another feature may look weak because its effect is dispersed across correlated variables. Attribution is only meaningful relative to a plausible counterfactual world.

This is where many explanations go wrong. They treat the observed values of the features as if they were neutral facts. But the values themselves are often part of the causal story. In SHAP terms, the contribution of a feature depends on the feature values, yet those values may be entangled with selection bias, missingness, or structural imbalance. Explanation then becomes a downstream ritual performed on upstream uncertainty.

SHAP tells us who gets credit, but not whether the stage was built correctly

SHAP has become attractive because it answers a deeply human question: why did the model say this? It decomposes a prediction into additive contributions, making the black box feel legible. This is especially powerful for tree based and linear models, where local and global patterns can be visualized with relative ease. But there is an important limit to this logic: a feature contribution is not the same thing as a causal effect, and a plausible explanation is not the same thing as a valid one.

Imagine a hospital readmission model that flags age, lab values, and prior admissions as important. SHAP can reveal that a patient’s elevated creatinine or repeated visits pushed the prediction upward. Helpful, yes. But if the dataset systematically under records outpatient care or social support, the explanation may overstate biological factors and understate structural ones. The model is assigning credit among the variables it can see, not among all the variables that matter.

This is why the phrase values of that feature matters so much. Explanation methods do not reason about abstract labels alone. They reason about actual feature values in context. A high blood pressure reading means something different for a young athlete than for an elderly patient with kidney disease. A high customer churn score means something different in a recession than in a stable market. The value is never just a number. It is a location in a latent data-generating process.

The deeper lesson is that explainability has two layers:

Attribution layer: Which observed features most affected the model’s prediction?
Structure layer: What hidden or missing information shaped those observed values in the first place?

Most explanation tools stay at layer one. But if the structure layer is wrong, the attribution layer can become a beautifully formatted lie.

Reconstruction is not the opposite of explanation, it is a prerequisite for it

The most interesting insight is that matrix completion and SHAP are not competing ideas. They are complementary steps in a larger epistemic pipeline. First, reconstruct the latent pattern as well as possible. Then, explain the prediction inside that reconstructed structure.

This is similar to editing a photograph. You cannot meaningfully adjust the contrast if the image file is corrupted. First you recover the underlying picture as faithfully as possible. Then you ask which pixels or regions carry the most signal. In causal panel data, matrix completion serves as the recovery step. It estimates the counterfactual outcomes that were never observed, allowing us to infer treatment effects or missing trajectories with greater discipline. In interpretability, SHAP serves as the allocation step. It tells us how the model uses the features once the underlying structure is in view.

The combination is powerful because it changes what counts as an explanation. Instead of asking, “Which observed variable had the largest contribution?” we can ask, “Which reconstructed state made this contribution necessary?” That is a much richer question. It moves us from blame assignment to structure discovery.

Here is a simple example. Suppose a retailer sees a sharp sales drop after a product launch. A standard explanation model might assign large negative credit to the launch date, inventory levels, and web traffic. But a matrix completion approach could reveal that the drop is part of a broader regional pattern tied to supply chain disruption, not just the launch itself. Once that latent pattern is reconstructed, the SHAP explanation changes. The same features may still matter, but now they are interpreted as symptoms of a larger hidden regime rather than isolated causes.

This is the difference between reading a sentence and reading the paragraph it belongs to. The sentence may be grammatically correct, but only the paragraph reveals its meaning.

Explanations are strongest when they are constrained by a recovered world, not just a fitted model.

A useful framework: three questions for every model you want to trust

To make this synthesis actionable, it helps to distinguish three layers of reasoning whenever you face incomplete, tabular, or panel data.

1. What is observed?

This is the familiar predictive layer. Which covariates, outcomes, and time points are present? Which records are missing? Which variables are proxies rather than direct measurements? Many modeling errors start here because practitioners treat the visible table as if it were the full reality.

2. What is reconstructible?

This is the latent structure layer. If the data were arranged as a matrix, what low-rank, repeating, or correlated patterns can be used to infer the missing entries? In panel data, this means looking for stable cross-sectional and temporal regularities that make counterfactual recovery possible. The goal is not perfect reconstruction, but reconstruction good enough to support causal reasoning.

3. What is attributable?

Only after the first two questions can we ask what a model is doing with the observed and reconstructed values. SHAP gives a disciplined way to allocate contribution, but the interpretation should be read against the backdrop of the reconstructed structure. Attribution then becomes a way to inspect how the model navigates the latent world, not just the visible table.

This three part framework prevents a common mistake: over trusting explanations that are mathematically precise but contextually thin. It also prevents the opposite mistake, which is to dismiss interpretability entirely because no feature attribution can fully capture causality. The right answer is not to abandon explanation. It is to nest explanation inside reconstruction.

The practical payoff: better decisions, not just prettier charts

Why does this matter outside the seminar room? Because every organization that uses data to make decisions faces hidden structure. Missingness is not an edge case. It is the norm. And when the missing structure is ignored, explanation tools can lead teams toward the wrong intervention.

Consider three settings:

Healthcare: A prediction model may identify medication history as a strong driver of readmission. But if social determinants and care access are under recorded, the true leverage may lie in post discharge support rather than clinical adjustment.
Finance: A credit model may emphasize income stability, but reconstructed panel patterns may show that neighborhood shocks or employment seasonality explain much of the risk. The apparent feature importance may be a proxy for a hidden cycle.
Operations: A demand forecast may make promotions look like the key driver of volatility. Yet matrix reconstruction of store level trajectories can reveal that supply interruptions, not promotions, created the largest discontinuities.

In each case, the failure mode is the same: a model is asked to explain outcomes in a world that was only partially observed. SHAP can still be useful, but its output should be read as the explanation of a model operating inside a partially reconstructed reality, not the explanation of reality itself.

That distinction matters because it changes the kind of action you take. If a feature is truly causal, you might intervene directly on it. If it is merely a proxy for an unobserved latent factor, you need a different lever. Without reconstruction, these two cases can look identical in a feature attribution plot.

Key Takeaways

Treat incomplete data as a structural problem, not just a missing data problem. The goal is to infer the latent pattern that makes the observed table coherent.
Use reconstruction before attribution when possible. If the underlying world is partially hidden, explanations based only on observed features can be systematically misleading.
Read feature importance as conditional, not absolute. A SHAP value tells you how the model used a feature within its observed context, not whether that feature is the true cause.
Ask what the feature value represents. The same numeric value can reflect different latent states depending on time, unit, and selection effects.
Prefer explanations that survive counterfactual scrutiny. If a feature remains important after reconstructing the missing structure, it is more likely to reflect something substantive.

The deepest shift: from explanation as ranking to explanation as recovery

The usual instinct is to think of explanation as a ranking problem. Which feature mattered most? Which variable came first? Which contribution was largest? But that framing quietly assumes the world was already fully visible. In real panel data, it often is not. The better framing is recovery: What latent pattern must exist for the observed values and model behavior to make sense?

That shift changes the role of interpretability. SHAP is no longer the final answer, but a diagnostic lens applied after the structure has been partially restored. Matrix completion is no longer just a missing data technique, but an epistemic tool for making explanation possible in the first place. Together, they point to a more mature view of modeling: not as a machine that announces truths, but as a system for piecing together a broken world.

The most important lesson may be this: the best explanations are not the ones that sound most certain, but the ones that are built on the strongest reconstruction of what was never directly seen. Once you understand that, feature importance plots look different. They stop being verdicts and start becoming clues. And that is a much better place to begin.