How to Build Security Systems That Cannot Be Patched

TL;DR

High-assurance security depends on eliminating the easiest viable attack, because one overlooked weakness can defeat otherwise strong protections. It is most necessary when systems cannot be patched, face intense attacks, or have severe failure consequences, but achieving it requires confronting complexity, human fallibility, fixed hardware, limited budgets, and the defender’s asymmetric burden.

Transcript

So in this talk, I'm gonna be looking at systems that aren't supposed to fail. And this isn't really the world that most of the security universe is focused on. Um, but I'll give you a little bit of background of what got me focused on this, then talk about where the failures that we struggle with and the problems that we're struggling with, with c... Read More

Key Insights

• Security is determined by the easiest successful attack, because an adversary needs only one exploitable weakness while a defender must protect every relevant path. Impressive cryptography, polished design, and strong individual components cannot compensate for an exposed dependency or overlooked implementation flaw.
• A security feature can look convincing while providing almost no protection. The promotional storage device had only one hundred possible combinations, used zero zero as its default password, and left data accessible while locked, illustrating the gap between attractive security marketing and effective controls.
• A formally analyzed protocol does not make the surrounding system secure. Proofs covering large parts of SSL missed a protocol flaw, while certificate authorities, configuration options, web servers, operating systems, and user decisions created additional dependencies whose failures could undermine the intended protection.
• Low-assurance security is a race between exploitation and recovery. Such systems contain defects and can be broken if an attacker invests enough effort, so their practical survival depends on whether vulnerabilities are discovered, exploited, detected, and patched before unacceptable damage occurs.
• Low assurance can be appropriate when attack consequences are minor, adversary resources are limited, or project complexity rules out stronger guarantees. The decision is therefore contextual, but it should reflect realistic consequences and attacker incentives instead of unsupported confidence that no defect will be found.
• Reliable recovery requires three capabilities: operators must know that the system is broken, an update mechanism must exist, and engineers must be available to perform the necessary re-engineering. If any capability is absent, a patch-dependent security strategy develops a serious weakness.
• Unpatchable systems require defects to be prevented before deployment. Payment smart cards are difficult to replace broadly, and ASIC defects become permanent after masks are made and chips are produced, even though fixed silicon also prevents attackers from modifying correct behavior.
• Defensive success is asymmetric because blocking ninety-nine percent of attacks can still be a complete failure if one remaining path succeeds. An attacker, by contrast, can fail repeatedly and still accomplish the objective after discovering a single usable entry point.

Questions & Answers

Q: What is high-assurance security?

High-assurance security is an approach for systems that are not supposed to fail and cannot safely depend on routine patching after deployment. It seeks confidence that software and hardware do not contain exploitable defects from the beginning. The need is greatest when consequences are serious, devices face malicious traffic, attackers apply sustained pressure, or deployed components cannot be replaced or updated easily.

Q: Why is security determined by the weakest link?

Security is determined by the weakest link because an attacker needs to find only one workable route into a system. A defender can stop ninety-nine percent of attempted attacks and still fail completely if one overlooked path permits compromise. Strong protocols, hardware, or cryptography therefore do not establish system security when another component, configuration, dependency, or human decision remains vulnerable.

Q: Why can a secure protocol fail to produce a secure system?

A secure protocol operates inside a larger collection of components and decisions. The discussion of SSL points to certificate authorities, complicated configuration choices, web servers, operating systems, and user behavior as additional dependencies. Even when proofs support important parts of a protocol, they may miss a flaw, and failures elsewhere in the system can prevent the protocol from delivering meaningful protection.

Q: When is low-assurance security an acceptable choice?

Low-assurance security can be acceptable when the consequences of compromise are limited, when attackers lack the resources or motivation to search deeply for defects, or when project complexity makes a stronger approach impossible. It remains a calculated risk rather than proof of security. The organization must also maintain dependable detection, updating, and engineering capabilities so discovered failures can be repaired.

Q: What does a patch-based security strategy require?

A patch-based strategy requires three essential capabilities. The operator must know that the system is broken, the system must provide a mechanism for distributing and applying an update, and engineers must remain available to redesign or repair the affected component. Missing any one of these capabilities creates a serious problem because known defects can persist without an effective path to recovery.

Q: Why do ASICs require high-assurance engineering?

ASICs require careful predeployment assurance because defects cannot be patched after the design is placed into silicon. Masks cost millions of dollars, and billions of chips may ultimately be produced from that fixed design. The same permanence has a benefit: if the hardware works correctly, attackers cannot readily modify its behavior. However, any embedded mistake can remain for the product’s lifetime.

Q: Why are payment smart cards difficult to secure through updates?

Payment smart cards are difficult to protect through ordinary recovery practices because a protocol defect cannot be corrected by simply pushing new software to every card. Replacing everyone’s credit card is also difficult. This makes prevention especially important, since the infrastructure lacks the convenient update process available to platforms that can regularly receive patches after vulnerabilities are discovered.

Q: How should engineers evaluate whether a security control works?

Engineers should evaluate what a control actually prevents, not how secure it appears. The storage-device example looked polished but offered only one hundred combinations, arrived with zero zero as the default password, and left data accessible while locked. A useful evaluation tests defaults, bypass paths, dependencies, user choices, and the easiest attack that defeats the claimed protection.

Summary & Key Takeaways

• High-assurance engineering begins with recognizing that a secure component does not guarantee a secure system. SSL could be supported by formal proofs while certificate authorities, confusing configuration choices, web servers, operating systems, and user decisions still introduced uncertainty. Security therefore has to be evaluated across every layer and dependency, not merely within the protocol.

• Low-assurance systems accept that defects exist and treat defense as a race among attackers, detection, and patching. This approach can be reasonable when consequences are limited, attackers lack sufficient resources, or complexity prevents stronger guarantees. It still requires knowing that a compromise occurred, having an update mechanism, and retaining engineers who can repair the system.

• High assurance becomes important when recovery is difficult or impossible. Payment smart cards, network-edge devices, anti-piracy technology, ASICs, and connected appliances illustrate different pressures. Attackers need to exploit only one overlooked weakness, while defenders must cover every meaningful path, making the easiest successful attack the practical measure of the entire system’s security.