Why Cybersecurity Projects Fail in Practice

TL;DR

Secure systems fail when elegant models conflict with normal user workflows or when builders mistake government requirements for dependable market demand. The Bell-LaPadula model reduced the amount of software that needed to be trusted, but routine tasks often required users to change security levels, bypass restrictions, or respond repeatedly to prompts, turning a theoretical breakthrough into a practical obstacle.

Transcript

Good afternoon. Welcome to the two twenty session. Our session is Lessons Learned: Fifty Years of Mistakes in Cybersecurity, and our speaker today is Steven Lipner, an executive director at SafeCode. Um, I'd ask all of you to please mute your phone so we can make this a more pleasurable experience. And then, uh, at the end of the session, there wil... Read More

Key Insights

• Cybersecurity projects can fail even when they are based on technically brilliant ideas. Lipner describes projects that appeared capable of changing the world but instead consumed substantial money and damaged reputations, prompting him to examine their shared assumptions and practical shortcomings.
• Multilevel security is a system for controlling access according to information classifications and user clearances. A person with a top secret clearance can access information at that classification and lower classifications, while someone with a secret clearance cannot access top secret material.
• The Bell-LaPadula model reduces trust requirements by constraining computer processes more strictly than cleared people. A top secret process can read or write top secret information and read unclassified information, but it cannot freely write information at the lower classification.
• The key benefit of Bell-LaPadula is that it changes the engineering problem from trusting all running software to certifying the software that enforces the security model. Lipner regarded this shift as a major conceptual breakthrough and a more tractable approach to secure-system construction.
• The Bell-LaPadula model worked in a Multics time-sharing system that became operational at the Pentagon for the Air Force. That success encouraged security practitioners and government organizations to view the model as a broadly applicable solution for multilevel security.
• The Orange Book incorporated the Bell-LaPadula model deeply into its security evaluation criteria. Its requirements encouraged commercial vendors to add classifications, clearances, mathematical models, formal specifications, verification, extensive testing, and other rigorous controls to their operating systems.
• Strict information-flow controls can interfere with ordinary communication. A user working at top secret who receives a secret email may need to log out and return at the secret level before replying, unless the system provides an exception that weakens the model.
• Pop-up fatigue occurs when security mechanisms repeatedly ask users to approve or deny actions. Lipner connects this problem to rigid security models and argues that increasing computer use made such workflow conflicts more frequent, turning protective controls into obstacles for users.

Questions & Answers

Q: What is multilevel security in computer systems?

Multilevel security is an approach for enforcing information classifications and user clearances within a computer system. The Defense Department model described by Lipner uses classifications such as top secret, secret, confidential, and unclassified. A user can access information at the level of that user's clearance or below it. For example, a secret-cleared user can access secret, confidential, and unclassified information, but not top secret information.

Q: How does the Bell-LaPadula security model work?

The Bell-LaPadula model controls how processes read and write information at different classifications. In Lipner's example, a top secret process can read or write a top secret file and can read an unclassified file. Its writing behavior is restricted so that top secret information is not transferred into an unclassified destination. These constraints allow the system to enforce information flow without trusting every program equally.

Q: Why was the Bell-LaPadula model considered a breakthrough?

The model was considered a breakthrough because it reduced the scope of the software that needed to be trusted. Instead of certifying every program operating in the system, designers could concentrate on the software responsible for enforcing the classification rules. Lipner describes this as a more tractable problem and says the idea encouraged organizations to build multilevel secure systems around a formal, enforceable model.

Q: Why did Bell-LaPadula create usability problems?

Bell-LaPadula created usability problems because legitimate work often crosses classification levels. Lipner gives the example of a user operating at top secret who receives a secret email and wants to reply. The model may require that person to log out and return at the secret level, or the system must create an exception. As computers became more common, these inconvenient situations also became more frequent.

Q: What is pop-up fatigue in security systems?

Pop-up fatigue is the effect of repeatedly asking users to approve or deny security-sensitive actions. Lipner uses the term while discussing models that impose strict controls on ordinary workflows. When routine activities continually trigger decisions or exceptions, the protection becomes burdensome and users repeatedly confront prompts. He presents this as one consequence of applying formal security rules without adequately accommodating how people actually use computer systems.

Q: What role did the Orange Book play in secure system design?

The Orange Book, formally described in the talk as the Trusted Computer Security Evaluation Criteria, strongly incorporated the Bell-LaPadula model. The National Security Agency developed the criteria in the early 1980s, with Lipner serving as an advisor. According to Lipner, the criteria encouraged or required vendors to build systems around that model because its creators believed it offered a general solution to multilevel security.

Q: How did commercial operating system vendors respond to multilevel security requirements?

By the early to mid-1980s, essentially all commercial operating system vendors were doing something to integrate the Bell-LaPadula model into their systems, according to Lipner. He names IBM, Honeywell, Burroughs, CDC, Univac, and Digital Equipment. Their projects incorporated security classifications and user clearances, reflecting the strong influence of government evaluation criteria and the assumption that formal multilevel security would have a viable market.

Q: What lessons can security architects draw from these failed projects?

Security architects should test whether a formal model supports routine user behavior and should treat projected market demand cautiously. Lipner's examples show that mathematical rigor and successful pilots do not guarantee a usable or commercially successful system. Controls that require frequent logouts, classification changes, exceptions, or approval prompts can obstruct legitimate work. Projects also become risky when organizations rely heavily on government claims about future demand.

Summary & Key Takeaways

• Steven Lipner traces his cybersecurity career from a 1970 MITRE assignment on multilevel security through work at Digital Equipment Corporation, Trusted Information Systems, Microsoft, and SAFECode. He focuses on expensive projects that failed despite ambitious goals, using those experiences to identify recurring errors in security architecture, product planning, and assumptions about customer demand.

• The Bell-LaPadula model was designed to enforce military information classifications in computer systems. It allowed a process to read information at its own or lower classification while restricting where that process could write. This reduced the challenge from trusting all application software to certifying the smaller body of software responsible for enforcing the security model.

• Although the model worked in a Multics system used by the Air Force at the Pentagon and became central to the Orange Book, it created serious usability problems. Common activities, such as replying to a lower-classification email, could require logging out, changing security levels, or creating exceptions, showing why mathematically rigorous security can still obstruct real work.