Safety-Critical Systems - Professor Martyn Thomas CBE

TL;DR

Computers in safety critical applications pose challenges in determining the causes of accidents, the reliability of software, and the definition of acceptable safety levels.

Transcript

I want to cover a range of topics that are current topics around the use of computers in safety related and safety critical applications I want to talk about what what actually causes accidents what do we mean when we say that something causes an accident about the way in which safety engineering has developed over the years the the social question... Read More

Key Insights

• 🦺 Safety engineering has evolved to address the challenges of making systems adequately safe in various contexts.
• 🦺 Software failures in safety critical applications are not random but systematic, making them different from physical failures.
• 💪 The absence of failures over time does not guarantee software reliability, and strong assumptions and evidence are needed to make claims about software reliability.
• 🦺 International safety standards, while flawed, provide recommendations for developing safety critical systems, but more work is needed to address cybersecurity threats.

Questions & Answers

Q: What are the challenges in determining the causes of accidents in safety critical applications?

Determining the causes of accidents in safety critical applications involves tracing back various factors, including human behavior, system design, maintenance, and external factors like weather conditions or infrastructure. It requires considering all possible contributing factors to understand the root cause accurately.

Q: What is the difference between random failures and systemic failures in software?

Random failures occur due to physical failures or wear and tear of components and are considered unpredictable. Systemic failures in software refer to design faults that always result in failure when specific conditions are met. Systemic failures are not random but can be traced back to design or programming errors.

Q: Can software reliability be determined based on the absence of failures over a certain period of time?

While the absence of failures over a period of time can provide some level of confidence, it does not guarantee software reliability. To make claims about the reliability of software, strong assumptions, auditable evidence, and a lack of changes to the software must be considered. Additionally, the operating conditions must be identical to those in the proven case.

Q: How do international safety standards address software reliability in safety critical systems?

International safety standards provide guidelines on development methods and practices based on the desired integrity level of the safety critical system. However, these recommendations are often based on flawed reasoning, and the connection between development methods and software reliability is not backed by scientific evidence.

Summary & Key Takeaways

• Safety engineering has developed over the years to address the challenges of making systems adequately safe in various contexts.

• The use of software in safety critical applications poses unique challenges as software failures are not random but systematic.

• Testing software can provide some level of confidence, but it cannot guarantee the absence of faults.

• International safety standards, while flawed, provide recommendations for developing safety critical systems.

• The value placed on a statistical life varies across different countries and industries.