Why Distributed Systems Fail When They Forget Where Atomicity Lives

Hatched by tfc

Jun 18, 2026

10 min read

2 views

86%

The hidden question behind modern architecture

What does it actually mean for a business action to finish?

That sounds like a simple question until software gets large enough to cross boundaries. A customer places an order, inventory must reserve stock, payment must clear, shipping must prepare a label, and analytics must record the event. If any one of those steps fails, did the order finish? Did it half finish? Is it still in progress? Was the business event successful even though one system disagrees?

This is where modern architecture gets dangerous. Teams often obsess over communication patterns, deployment models, or infrastructure choices, but the deeper problem is more fundamental: where does atomicity live? If you cannot answer that clearly, you get systems that look decentralized but behave like a fragile tangle of hidden dependencies.

The temptation is to assume the answer is serverless, microservices, or events. But those are not answers. They are execution styles. The real answer is architectural discipline around business boundaries, team boundaries, and the smallest unit of work that can be trusted.

The most important thing a system can know is not how to scale, but what counts as one complete act.

Atomicity is not a database concern anymore, it is a design responsibility

In traditional systems, atomicity was often treated as a storage problem. A transaction either committed or rolled back. That model gave engineers a comforting illusion: if the database is consistent, the application is probably fine. But once software spreads across services, queues, functions, APIs, and event handlers, atomicity stops being a feature of the database and becomes a question of coordination.

That is why the idea of a Unit of Work matters so much. At its core, it is an abstraction over atomic operations. It says that a set of changes should be treated as one meaningful whole. Either the work completes, or it does not. That sounds technical, but the real power is conceptual: it teaches the system to recognize the difference between a collection of actions and a single business commitment.

Consider a simple retail example. A user clicks “buy.” Behind the scenes, several steps need to happen: create the order record, reserve inventory, authorize payment, and emit a confirmation event. If those steps are scattered across unrelated mechanisms with no clear unit of work, the system may end up with an order that exists without stock, stock that is reserved without payment, or a payment that succeeded for an order that never became real. That is not just a technical bug. It is a business contradiction.

The deeper lesson is that atomicity is not merely about avoiding partial writes. It is about protecting meaning. A business action should either become a durable fact or remain an unrealized intent. Anything in between creates ambiguity, and ambiguity is where distributed systems decay.

This is why many organizations get into trouble when they scale quickly into serverless or event-driven designs without first establishing the shape of their work. They gain elasticity, but lose coherence. They gain speed, but lose the ability to say what one completed action even is.

Serverless does not create complexity, it reveals whether you already had it

The promise of serverless is seductive: smaller units, faster delivery, lower operational burden, automatic scaling. But if enterprises adopt it as a default architecture instead of a tool chosen for a specific domain problem, they often create a Ball of Serverless Mud: a distributed monolith disguised as agility.

That phrase captures an important failure mode. The danger is not that functions are small. The danger is that their relationships become large, hidden, and uncontrolled. One function publishes an event that triggers three others, which each call more services, which each rely on assumptions no one documented. Soon the system still has all the coordination cost of a monolith, plus the debugging pain of a distributed network.

This happens because teams start with infrastructure instead of meaning. They ask, “Can this be serverless?” when the more useful question is, “What is the domain boundary, and what should be the unit of work inside it?” If you do not know the answer to that second question, serverless simply multiplies the number of places where uncertainty can hide.

The critical shift is to treat serverless as a technology enabler, not a design philosophy. A function may be the right implementation for a specific business capability, but only if the capability itself has been carefully defined first. In other words, the architecture should emerge from the domain, not from the deployment model.

This is why principles such as domain-first, team-first, API-first, microservices-first, event-driven-first, and only then serverless-first matter. They are not slogans in a checklist. They are a sequence of constraints that prevent confusion from becoming architecture.

A useful analogy is city planning. If you start by choosing the kind of streetlight you want, you will build a confusing city. If you start with neighborhoods, traffic flows, utilities, schools, and commuting patterns, then the streetlight choice makes sense. Serverless is a streetlight decision. Atomicity and domain boundaries are neighborhood decisions.

The real boundary is not technical, it is operational

The most overlooked insight in modern architecture is that domain boundaries and atomic boundaries should align as closely as possible. This is where the Unit of Work and domain-first thinking meet. The Unit of Work says, “These changes belong together.” Domain-driven thinking says, “This business capability is a coherent unit of meaning.” When those overlap, systems become easier to reason about, test, and evolve.

When they do not overlap, trouble starts quickly. Suppose a company has a marketing team, a fulfillment team, and a billing team, all pushing changes into the same event stream with different interpretations of success. One team thinks an order is complete when it is paid. Another thinks it is complete when it ships. A third thinks it is complete when the customer receives a confirmation email. That is not a technical disagreement. It is a broken model of the business.

A well designed system does not merely move messages around. It encodes a shared agreement about what counts as one action. That agreement needs to be visible in the code, the API, the team structure, and the operational playbook. If a team cannot explain the lifecycle of a business action in one sentence, then the architecture probably cannot support it safely.

The best distributed systems do not distribute responsibility for meaning. They distribute responsibility for execution.

This distinction matters because many organizations confuse independence with fragmentation. They break a monolith into services or functions and assume they have achieved autonomy. But autonomy is not just about running separately. It is about having a clear, bounded purpose, a clear interface, and a clear completion condition.

That is why team structure is not an afterthought. Teams are often the real architecture. If one team owns reservation, another owns payment, and another owns order lifecycle, then each team needs a precise contract about what state changes are allowed, when an action is complete, and how failures are reconciled. Otherwise the code will reflect the org chart’s confusion.

In practice, this means the most important design artifact is not the cloud diagram. It is the definition of the business transaction. What begins it? What steps belong inside it? What is guaranteed at the end? What compensation is acceptable if it fails halfway? Those questions determine whether your system is resilient or merely busy.

A mental model: every business action has a center of gravity

One way to unify these ideas is to think of each business capability as having a center of gravity. The center of gravity is the place where meaning, ownership, and atomicity naturally belong.

If the center of gravity is clear, the architecture becomes simpler:

The domain model knows what a completed action means.
The Unit of Work knows what must commit together.
The API exposes a boundary that reflects business intent.
Events are used to notify others, not to define the business itself.
Serverless functions become implementation details around the edges.

If the center of gravity is unclear, the system starts orbiting chaos. Events become business logic. Functions become hidden workflows. Retry logic becomes state management. Monitoring becomes archaeology.

Here is a concrete example. Imagine a travel booking platform. A customer books a flight, hotel, and car rental in one flow. If every piece is treated as an independent serverless function with its own completion rules, the system may technically succeed while delivering a terrible experience. The customer gets a flight confirmation, but the hotel fails. Is the booking complete? Maybe yes for one team, no for another.

A better design starts by identifying the business atom. Is it the full trip? A single supplier reservation? A payment authorization? Once that is defined, the Unit of Work can protect that atom, while events propagate outcomes to downstream systems. The point is not to force all details into one transaction. The point is to define the smallest business unit that must be coherent before anything else can safely react.

This is where many event driven systems go wrong: they confuse notification with completion. An event says something happened. It does not magically prove that the thing is finished in the business sense. That proof must come from a clear boundary of ownership and atomicity.

A good rule is this: if a system cannot tolerate seeing the intermediate state, that state should not be exposed as a first class business fact.

Synthesis: start with what must be true, then choose the tool

The intersection of these ideas points to a simple but demanding thesis: architectures fail when they choose communication style before defining business completion.

That is why serverless often disappoints when adopted as a trend. It encourages decomposing everything into functions before the organization knows where the real units of work live. The result is a large number of tiny parts with no shared understanding of what success means. The system scales mechanically while becoming conceptually incoherent.

The more mature approach reverses the order:

Define the business domain clearly. What is the capability, what outcome matters, and who owns it?
Identify the atomic business action. What has to happen together for the business to consider the action complete?
Match the team boundary to the domain boundary. Who can reason about this action end to end?
Choose APIs, events, and serverless functions as implementation choices. Not as ideology, but as mechanisms.

This is not anti serverless. It is pro coherence. Serverless can be excellent when the work is already well bounded, the contracts are clear, and the failure modes are understood. In fact, serverless can make a well designed unit of work easier to operate, because the infrastructure disappears and the business semantics become easier to see.

But if the unit of work is unclear, then the serverless estate becomes a fog machine. Everything looks lightweight until failure arrives, and then every hidden dependency becomes visible at once.

The real architectural skill, then, is not decomposition for its own sake. It is meaningful decomposition. The parts should be smaller, yes, but also more honest. Each part should know what it owns, what it promises, and what it must never pretend is complete before it really is.

Key Takeaways

Define completion before you define components. Ask what must be true for a business action to count as finished, then design around that boundary.
Treat atomicity as a business concept, not just a database feature. The Unit of Work is a way of protecting meaning, not merely grouping writes.
Use serverless as an implementation tool, not an architectural ideology. If the domain is unclear, serverless will amplify confusion rather than solve it.
Align team boundaries with domain boundaries. A team that cannot explain one business transaction end to end is a warning sign.
Separate completion from notification. An event can announce change, but it should not be mistaken for proof that the business has reached a coherent state.

Conclusion: the future belongs to systems that know what one thing is

The biggest mistake in modern architecture is thinking that distribution is the main challenge. It is not. The real challenge is preserving the identity of a business action when the software around it becomes fragmented.

A well designed system does not merely move faster or scale farther. It knows where one meaningful thing ends and another begins. That is what the Unit of Work protects. That is what domain-first architecture clarifies. And that is what serverless, used wisely, can help execute.

So the next time a team asks how to break a system apart, ask a harder question first: what is the smallest complete truth the business needs to preserve? If you can answer that, the architecture becomes an expression of clarity. If you cannot, no amount of cloud machinery will save you from the ball of mud you are already building.