Why AI May Beat Humans at Software Architecture

TL;DR

Most architectural failures trace back to lost context, not bad judgment. Information needed to prevent problems exists but is spread across too many files, people, and moments for any one mind to hold. Because human working memory caps at four to seven chunks, AI's larger context windows give it a structural advantage at spotting entropy-driven decay in large codebases.

Transcript

AI might be better at software architecture than humans. Not because AI is smarter, but because humans are structurally incapable of the kind of vigilance that good scaled technical architecture requires. That's a very strong claim and it cuts against everything we've been told typically. So the conventional wisdom for a couple of years now has bee... Read More

Key Insights

• Architectural failures are almost never caused by bad judgment. The root cause is lost context: the information needed to prevent the problem existed but was spread across too many files, people, and moments in time for any single mind to hold at once.
• Performance problems are entropy problems, not technical problems. Ding, who spent roughly 7 years at Vercel and opened around 400 performance-focused pull requests, argues entropy emerges from the mismatch between human cognitive limits and the scale of modern software.
• Human working memory can hold only four to seven chunks of information. This is a structural limitation, not a training problem, and it caps how many concerns an engineer can weigh simultaneously during architectural reasoning.
• Systems rot through the accumulation of local reasonable decisions. Every individual change makes sense and passes review, yet together they create messes no single person saw coming, like a tragedy of the commons written as architectural failure.
• Abstractions conceal cost. A clean reusable popup hook that adds a global click listener means 100 instances fire 100 callbacks on every click anywhere on the site, and nothing in the codebase warns the next engineer reusing it.
• Fragile abstractions break silently. Adding an object parameter to a cached function creates a new object reference each call, so the cache never hits, and neither type safety nor linters catch it until someone profiles the app months later.
• AI has structural advantages in attention span, memory, and holding an entire codebase in mind. As context windows grow larger and searchable, evaluating a single line change against the whole system becomes a viable model for AI agents.
• Good intentions do not scale. Entropy wins not through malice or incompetence but because the system allows degradation, and telling engineers to simply pay more attention cannot overcome a structural mismatch of scale.

Questions & Answers

Q: Why does AI have a structural advantage over humans in software architecture?

The speaker argues AI's advantage comes not from being smarter but from attention span, memory, and the ability to hold an entire codebase in mind while evaluating a single line change. Humans are structurally incapable of the vigilance that scaled architecture requires, because most architectural failures stem from lost context rather than bad judgment. As context windows grow larger and searchable, holding the whole system in view becomes a viable model for AI agents in 2026 and beyond.

Q: What is the real root cause of most architectural failures?

The root cause is almost never bad architectural judgment. It is lost context. The information needed to prevent the problem did exist, but it was spread across too many files, too many people, and too many moments in time for any single human mind to hold at once. Original architectures are often fine, engineers are competent, and code reviews are thorough, yet systems still rot between the initial design and the daily reality of shipping features to production.

Q: What does it mean that performance problems are entropy problems?

Ding, who spent roughly 7 years at Vercel and opened around 400 performance-focused pull requests, argues that performance problems are not technical problems but entropy problems. Codebases grow exponentially through dependencies, state machines, async flows, and caching layers, faster than any individual can track. Entropy is not something you can patch with faster frameworks or smarter linters. It is a systemic problem emerging from the mismatch between human cognitive architecture and the scale of modern software systems.

Q: How does human working memory limit architectural reasoning?

Research shows humans can hold only four to seven chunks of information in their heads at once. This is a structural limitation, not something experience or training can overcome. Good architectural reasoning requires holding many concerns simultaneously, including performance, security, maintainability, existing patterns, and downstream effects on other teams. A moderately complex decision might involve a dozen relevant considerations, so humans use abstractions to cycle through them, but those abstractions only scale so far when done in one head.

Q: What is an example of an abstraction that conceals cost?

The speaker describes a reusable popup hook that looks clean and adds a global click listener to detect when users click on a popup. The abstraction hides that every single instance adds a global listener. With 100 popup instances across an application, that means 100 callbacks firing on every click anywhere on the website. The technical fix is easy, but the systemic problem is that nothing in the codebase prevents the pattern from spreading, and the cost stays invisible until users complain about sluggish performance.

Q: Why do caching optimizations break silently?

An engineer extends a cached function by adding an object parameter, a reasonable change. The code compiles and tests pass, but every call creates a new object reference, which means the cache never hits and is completely broken silently. The knowledge to do it correctly exists in the documentation, but nothing enforces that documentation. Type safety does not help and a linter will not catch it. The cache quietly stops working, and nobody notices until someone profiles the app months later.

Q: How can a single change hurt performance in a large function?

The speaker gives the example of adding a coupon check to a thousand-line function that processes orders, built by multiple people. The engineer adds an await for coupon validation to solve a local problem, but the check now blocks everything below it, creating a waterfall where sequential operations could have run in parallel. The engineer cannot see the global asynchronous flow because it is spread across thousands of lines written by people who no longer work there, making the optimization opportunity invisible.

Q: Does the speaker believe AI will replace software architects?

No. The speaker explicitly states this is not a polemic about AI replacing architects, and that architects still have a key role. The argument is an attempt to reason backwards from the core principles underlying architecture to understand where cognitive advantages actually lie for humans versus AI. Humans remain irreplaceable for novel decisions and trade-offs, while the opportunity is designing AI partnerships that address the entropy humans were always going to lose to, rather than replacement.

Summary & Key Takeaways

• The conventional wisdom is that AI is bad at architecture because it requires holistic thinking and accumulated wisdom. But when engineers describe failures like degraded performance or broken caching, the root cause is almost never bad judgment. It is lost context spread across too many files, people, and moments for one mind to hold.

• Ding, after roughly 7 years at Vercel and around 400 performance pull requests, argues performance problems are entropy problems. Codebases grow faster than any individual can track, context fades as engineers switch features, and knowledge dilutes as teams scale. As he put it, you cannot hold the design of the cathedral in your head while laying a single brick.

• Four production examples show clean local decisions causing systemic decay: a popup hook adding global listeners, a cache broken by a new object reference, a coupon check blocking a thousand-line checkout waterfall, and needless memoization. Because human working memory holds only four to seven chunks, AI agents with large context windows may hold structural advantages here, though architects remain essential.