Why Good Design Is Really About Surviving Failure

Kai Nguyen

Hatched by Kai Nguyen

Aug 04, 2026

11 min read

91%

0

The hidden question behind every clean system

What if the real test of design is not whether things work when everything goes right, but whether they remain understandable and recoverable when something goes wrong?

That question sits beneath two ideas that are often treated as separate: clean object oriented design and distributed transaction management. One speaks the language of classes, interfaces, and responsibilities. The other speaks the language of services, events, and compensating actions. But both are wrestling with the same fundamental problem: how do you build systems that can change, fail, and still remain coherent?

In small codebases, failure is often local and visible. A method breaks, a class gets too large, a dependency becomes tangled. In distributed systems, failure is more dramatic. A payment succeeds but inventory reservation fails, or a user account is created but the welcome email never sends. Yet the deeper challenge is the same in both worlds. Complexity grows until the system becomes hard to reason about, and then the cost of every change rises.

That is why the most useful way to think about design is not as a style preference, but as a strategy for preserving meaning under pressure.


SOLID and Saga are answering the same design anxiety

At first glance, object oriented design principles and distributed transaction patterns seem to live in different universes. One concerns software structure inside a codebase. The other concerns coordination across services. But both are trying to solve a single recurring anxiety: how to keep responsibilities small enough that failure does not spread uncontrollably.

SOLID does this by narrowing the blast radius of change. If a class has one reason to change, then a new feature or bug fix does not force unrelated logic to shift with it. If dependencies point toward abstractions, then lower level details can be replaced without rewriting the world. In other words, SOLID makes local change survivable.

Saga does something remarkably similar at system scale. Instead of pretending that a distributed system can behave like a single ACID transaction, it accepts that each service owns its own data and performs its own local transaction. If the larger business process fails halfway through, the system does not roll back magically. It compensates. That is a very different philosophy. It says: do not eliminate failure, design for recovery.

This is the first important connection: both approaches reject brittle coupling. Both replace the fantasy of total control with a more realistic model of constrained responsibility.

Good design is not the absence of failure. It is the ability to contain failure without losing the plot.

That idea matters because it changes the goal. You are not trying to build a perfect machine. You are trying to build a system that can absorb the inevitable imperfections of real life.


The real enemy is not complexity, it is entanglement

Complexity is often blamed for bad software, but complexity alone is not the villain. A payroll system is complex by nature. A marketplace with sellers, buyers, payments, fraud checks, shipping, refunds, and tax rules will always be complex. The problem begins when complexity becomes entanglement.

Entanglement means that one decision ripples through unrelated parts of the system. A class meant to calculate discounts also sends emails. A checkout service also updates loyalty points, inventory, and analytics directly. A seemingly small change then becomes an excavation project because no one can alter one part without disturbing five others.

SOLID principles are a map for resisting entanglement at the code level. Saga patterns are a map for resisting entanglement at the architecture level. Both ask the same question in different dialects: where does one responsibility end and another begin?

A useful mental model here is to imagine a city.

A well designed city does not put every function into one giant building. Hospitals, schools, power, water, transport, and housing all have distinct jobs. When a water pipe bursts, the entire city should not collapse. When a school closes, the power grid should continue functioning. Each subsystem has its own logic, its own maintenance cycle, and its own boundaries.

Software systems need the same urban planning mindset. If every service or class is a neighborhood with clear borders, then failure is an incident. If every component is connected by hidden dependencies, failure becomes a contagion.

This is why simple measures like “keep classes small” or “split services” are not enough on their own. Smallness is not the goal. Separation of reasons to change is the goal. A tiny class that secretly depends on ten things is still tangled. A service boundary that merely forwards calls is still a facade. The deeper task is to create meaningful seams.


Compensation is the ethics of distributed design

Saga introduces a striking idea that object oriented design often leaves implicit: sometimes the correct response to a failed action is not to undo it perfectly, but to repair the business state by performing a different action.

That sounds technical, but it is philosophically important. Compensation acknowledges that not every operation is reversible in a literal sense. A shipping label may be purchased, a fraud check may be logged, a message may be delivered, an external API may have already observed the event. Distributed systems live in a world where time has consequences.

This resembles a principle that good modular design also teaches, though more quietly. When a responsibility is isolated properly, you do not need global surgery to fix a local issue. You can substitute behavior, redirect dependencies, or replace a module without rewriting the system from scratch. The change is not erased. It is absorbed.

Consider an online store.

A customer places an order. Inventory is reserved, payment is authorized, shipping is initiated. If shipping fails after payment succeeds, the system should not pretend nothing happened. Instead, it might void the payment, release inventory, and mark the order as failed. This is compensation in action.

Now compare that to a well designed class hierarchy. If a pricing strategy changes, you should be able to swap the pricing object without altering checkout logic, cart logic, or notification logic. The code does not roll back the world. It contains the effect of the change.

Both cases reveal a deeper design ethic: responsible systems do not require perfection from every step, only coherence across the whole process.

This is a powerful shift because it moves the conversation away from idealized correctness and toward operational honesty. A design that assumes nothing can fail often becomes fragile. A design that assumes failures will occur can remain usable.


A useful framework: local integrity, global coordination

To connect these ideas into something practical, it helps to use a two layer framework.

1. Local integrity

At the smallest meaningful unit, a class or service should be internally coherent. It should do one thing, or one tightly related set of things, and do them well. It should have clear dependencies and minimal hidden side effects. This is where SOLID matters most.

Local integrity means the unit can be understood, tested, and changed without needing the entire system in your head. It is not just about elegance. It is about reducing cognitive load and minimizing accidental breakage.

2. Global coordination

At the system level, units must cooperate across boundaries. This is where Saga matters most. The system should define a business process as a sequence of local actions, plus a plan for what happens when one step fails. The coordination mechanism can be an orchestrator or a choreography of events, but the key is that the overall workflow remains understandable even though no single database controls everything.

Global coordination means the business process survives partial completion. It may pause, retry, compensate, or route around failure, but it does not rely on an impossible illusion of atomicity.

The brilliance of this framework is that it reveals a symmetry. SOLID protects the integrity of the part. Saga protects the integrity of the whole. If you neglect the first, your components become unmanageable. If you neglect the second, your architecture becomes brittle.

The mistake many teams make is optimizing one layer while ignoring the other. They write clean classes inside a messy architecture, or build an elegant event driven architecture on top of entangled internal modules. Either way, the design leaks.

A system is only as resilient as its boundaries are meaningful.


Why orchestration and choreography mirror object design tradeoffs

Saga offers two main coordination styles: orchestration and choreography. That distinction is not just about distributed systems. It echoes a classic tension in software design: centralized control versus decentralized responsibility.

In orchestration, a central coordinator tells each participant what to do and when. This is easier to understand at first because the flow is explicit. You can read the process from top to bottom, like a script. But the orchestrator can become a point of complexity if it grows too large or starts owning too much business logic.

In choreography, each service reacts to events and emits new ones. There is no central conductor. The overall process emerges from local rules. This can scale beautifully, but it can also become hard to trace if the event graph is not carefully documented.

That tradeoff mirrors design choices inside code. A monolithic class that coordinates everything may be simple to start with, but it tends to accumulate too many reasons to change. A fully decentralized set of classes may preserve separation, but without a clear protocol the behavior becomes opaque.

The lesson is not that one style is always better. The lesson is that control should live at the right level of abstraction. If a decision is business critical and needs clear visibility, orchestration may be right. If the workflow is naturally event driven and independent, choreography may be better. Likewise, some responsibilities should be pulled into one object, while others should be split across collaborators.

Design is not about choosing centralization or decentralization in the abstract. It is about locating authority where it causes the least coupling and the most clarity.

A simple analogy: a kitchen.

A fine dining kitchen has a head chef who coordinates timing, but individual stations handle their own responsibilities. The sauce station does not need to know the details of plating, and the pastry station does not need to interfere with the grill. The whole meal is coordinated, but not micromanaged. Good software wants that same balance.


The deeper thesis: design is the art of making recovery legible

Once you connect SOLID and Saga, a more ambitious thesis appears: the highest purpose of design is not cleanliness, but recoverability.

Why? Because software exists in time. Requirements change. Dependencies fail. Networks split. Teams grow. External APIs misbehave. Humans make mistakes. A design that only works when conditions remain ideal is not robust, only lucky.

Recoverability has two dimensions.

First, the system must be able to recover technically. Failures should be localized, observably logged, and either retried or compensated. Second, the system must be recoverable cognitively. Engineers should be able to understand what happened, where responsibility lives, and how to make changes safely.

This is where clean class design and distributed coordination meet. SOLID reduces the mental cost of recovery because each component has a clear role. Saga reduces the operational cost of recovery because each step of a business process has an explicit failure path.

The best systems are not the ones that never break. They are the ones that make breakage narratable. You can tell the story of what happened, why it happened, and what the system did in response.

That narrative quality is underrated. When a bug occurs, the most dangerous question is not “what failed?” but “what else might this have affected?” A good design should make the answer narrow and discoverable.

Think about a bank transfer. If the debit side completes but the credit side fails, the system needs to know exactly which state transitions happened and how to compensate. If the transfer logic is embedded in a single opaque function that touches every subsystem, the story becomes impossible to reconstruct. If each step is isolated, the story stays legible.

That is why architecture is not just structure. It is a theory of causality.


Key Takeaways

  1. Design for failure, not fantasy. Assume that steps will fail, dependencies will drift, and recovery will be necessary.

  2. Separate reasons to change, not just code. Small units are useful only when they represent meaningful boundaries of responsibility.

  3. Use local integrity and global coordination together. Clean classes are not enough if the system flow is tangled, and smart workflows are not enough if the internals are messy.

  4. Prefer explicit recovery paths. Whether it is a compensating transaction or a swappable dependency, make the fallback visible and intentional.

  5. Choose control at the right level. Centralize only what benefits from explicit oversight, decentralize what benefits from autonomy and isolation.


The final reframing: good design is a memory of how the system survives

We often talk about software design as if it were primarily about order, elegance, or simplicity. Those are useful outcomes, but they are not the deepest measure.

The deeper measure is this: when the system encounters the unexpected, does it remain intelligible?

SOLID teaches that inside a codebase, responsibilities should be clean enough that change does not infect everything. Saga teaches that across services, business processes should be resilient enough that partial failure does not destroy the whole. Together they suggest a unifying principle: the best design is the one that preserves meaning under disruption.

That is a more demanding standard than writing code that merely works. It asks us to think like engineers of resilience, not just builders of features.

So the next time you split a class, define a service boundary, or add a compensating step, ask a better question than “Is this elegant?” Ask: If this fails, can the system still tell the truth about what happened?

If the answer is yes, you are not just designing software. You are designing survivability.

Sources

← Back to Library

Hatch New Ideas with Glasp AI 🐣

Glasp AI allows you to hatch new ideas based on your curated content. Let's curate and create with Glasp AI :)

Start Hatching 🐣