Scott Borg on Measuring Cybersecurity Risk

TL;DR

Cybersecurity performance should ultimately be measured by risk, defined as likely losses over a given period, ideally expressed as an annualized expected loss estimate. When reliable risk estimates are not possible, organizations can measure how much their defenses raise attacker costs through greater expertise and time requirements, then prioritize protections around the business activities, information, designs, and records that create the most value.

Transcript

Hi, this is Erica Czuchowski here at the RSA Conference. I'm here today with Scott Borg, who is the director of the US Cyber Consequences Unit. And we're talking a little bit today about security metrics because that's what, uh, Scott is doing a talk about. So Scott, some people might say security metrics are hard, if not impossible. Um, can you te... Read More

Key Insights

• Risk is the central security metric because every other measure should feed into an estimate of likely losses over a given interval. Borg says the preferred expression is an annualized expected loss estimate, although the estimate may need to remain rough.
• Security success is the reduction of expected loss. From Borg's perspective, other aspects of a security professional's work matter only insofar as they help bring the organization's risk estimate down under relevant operating and attack conditions.
• Business value is concentrated rather than distributed evenly across an organization. A few activities often provide competitive advantage or account for profit, while many other activities matter less, so security teams must identify the systems supporting the most valuable areas.
• Valuable business assets are attractive attack targets because they can include vital information, key designs, and precious customer records. The same assets that contribute most strongly to competitive advantage are therefore among the assets an organization most needs to protect.
• Reliable risk estimates require knowledge of consequences and threats. Security professionals need to understand where value and liabilities exist, who the attackers are, how those attackers can profit, and how their business models are changing.
• Organizational limits can prevent effective risk measurement. Borg says many cybersecurity professionals are not given enough scope or authority to understand business consequences, collect sufficient attacker information, and perform threat analysis well enough to produce a good estimate.
• Attacker cost is the most useful metric when a credible risk estimate is unavailable. It can be approximated through the expertise required and the number of hours, days, or weeks an attacker must apply that expertise to complete an attack.
• Attacks become unattractive when their costs exceed their gains. An organization can also improve its relative position by making itself more expensive to attack than other organizations, although raising costs for an attack nobody intends to conduct can waste resources.

Questions & Answers

Q: What is the most important cybersecurity metric?

Risk is the most important cybersecurity metric because other security measurements should ultimately feed into it or help reduce it. Borg defines risk as the likely losses an organization will experience during a given interval. Ideally, it should be expressed as an annualized expected loss estimate, even if the available information supports only a rough but credible number.

Q: How should an organization measure cyber risk?

An organization should estimate the losses it is likely to experience over a defined period under different conditions. Those losses do not have to be expressed only in money because they can involve lives or equipment. Borg says the preferred form is an annualized expected loss estimate, with security work evaluated according to whether it lowers that number.

Q: How can cybersecurity metrics connect to business value?

Cybersecurity metrics connect to business value when security professionals identify where and how the organization creates value, along with where it could face its greatest liabilities. They should determine which business areas provide competitive advantage or profit and then understand the systems supporting them. This makes protection efforts relevant to the assets and operations that matter most.

Q: Which business assets should security teams prioritize?

Security teams should prioritize the assets connected to the limited areas where the organization creates its competitive advantage or profit. Borg identifies vital information, key designs, and precious customer records as examples. These assets are important both because the business depends on them and because attackers are likely to pursue information and systems tied to valuable activities.

Q: Why is producing a good cyber risk estimate difficult?

Producing a good risk estimate is difficult because many cybersecurity professionals do not know where their organization creates value or understand consequences well enough. They may also lack sufficient information about attackers, including who they are, how they profit, and how their business models are changing. Borg also says security teams often lack the necessary scope and authority.

Q: What should security teams measure when risk estimates are unavailable?

Security teams should measure attacker costs when they cannot produce a credible risk estimate. A useful rough estimate considers the expertise required for an attack and how many hours, days, or weeks an attacker must apply that expertise to succeed. Security controls can then be evaluated according to how much they increase those requirements and make attacks harder to justify.

Q: How does increasing attacker cost improve security?

Increasing attacker cost improves security by forcing adversaries to use more expertise and spend more time carrying out an attack. Borg says a mid-sized business can often raise attacker costs by a factor of 10 and sometimes by a factor of 100. Even a well-resourced attacker does not have an unlimited budget, so large cost increases can constrain its choices.

Q: When has an organization raised attacker costs enough?

An organization has raised attacker costs enough in absolute terms when the cost of conducting the attack exceeds the expected gain, removing the reason to proceed. It can also succeed in relative terms by making its systems more costly to attack than other organizations. However, attacker cost remains less useful than risk if defenders harden against attacks nobody wants to carry out.

Summary & Key Takeaways

• Scott Borg argues that security metrics are useful because they show whether security work is accomplishing anything and moving in the right direction. Every metric should ultimately contribute to understanding risk, which he defines as the likely losses an organization will experience within a specified interval under different conditions.

• Risk measurement requires security professionals to understand where their organization creates value and where it faces its greatest liabilities. Because only a few business areas may provide competitive advantage or generate profit, defenders should identify the systems, designs, information, and customer records connected to those especially valuable areas.

• Many cybersecurity professionals lack the organizational scope, authority, consequence data, and attacker intelligence required to produce credible risk estimates. In that situation, attacker cost is the most useful alternative metric. Defenses should increase the expertise and time an attacker needs until the attack becomes unattractive or less appealing than other targets.