How to Predict Which Vulnerabilities Get Exploited

TL;DR

Prioritize vulnerabilities by forecasting which ones attackers are likely to exploit, rather than relying only on severity scores or observed attacks. Narrowly scoped machine learning models can use historical vulnerability and exploit data to create actionable predictions, helping teams compare remediation effort, risk reduction, and the effectiveness of each action.

Transcript

Welcome everyone. My name is Michael Reitman. I'm the chief data scientist at Kenna Security. This talk is about issuing predictions for when a vulnerability will be exploited. But the takeaways from this talk are entirely different. Security is often a pendulum between prevention and detection and response, and there are good reasons for that. Tec... Read More

Key Insights

• Machine learning is most useful when it addresses a narrowly defined security question. Content expertise is essential for framing that question correctly, and a simple model can produce substantial business value when its prediction target corresponds to an operational vulnerability-management decision.
• Vulnerability analysis has three relevant stages: retrospective description, real-time processing, and prediction. Retrospective methods categorize previously observed flaws, real-time systems collect current exploitation signals, and predictive methods use historical observations to generate new data about events that have not happened yet.
• CVSS is a point-in-time assessment of a software flaw and its potential impact. It provides useful categorical information such as access complexity, authentication method, and confidentiality impact, but it says comparatively little about the actual risk that a vulnerability poses to an organization.
• Successful exploitation affects only a small portion of recorded vulnerabilities. Among roughly 120,000 vulnerabilities in the National Vulnerability Database, about 95,000 are confirmed, perhaps 20,000 have proof-of-concept or black hat exploit code, and only about two percent show successful exploitation events.
• Exploitation probability increases as time passes after vulnerability publication. The presented data shows an inflection around 40–60 days, but the probability is already non-zero at release and rises to about 25 percent by day 10 within the population being examined.
• CVSS 10 remediation performs better than random selection but remains inefficient. Remediating all CVSS 10 instances across the cited data set reaches about 11.5 percent of vulnerabilities associated with successful exploitation, compared with about six percent for random selection.
• Exploit availability is a stronger prioritization signal than maximum severity alone. Vulnerabilities appearing in both Exploit-DB and Metasploit reach a 36 percent remediation hit rate while representing about 15 percent of vulnerabilities, making roughly one in three resulting actions effective.
• Remediation strategies should balance work effort against measurable risk reduction. Teams should examine how many vulnerabilities a rule requires them to fix, how much exploitation risk those fixes address, and how often each remediation action targets a vulnerability that attackers actually exploit.

Questions & Answers

Q: How can machine learning predict vulnerability exploitation?

Machine learning can predict exploitation by taking historical vulnerability characteristics and related event data, then generating a new data set representing likely future outcomes. The problem must be tightly scoped, such as predicting whether attackers will develop or use an exploit for a particular vulnerability. Domain expertise is essential because it determines which question is operationally useful and which observations should inform the model.

Q: Why is prediction important for vulnerability prevention?

Prevention requires acting before exploitation is observed. Retrospective scores describe known characteristics, while real-time feeds identify activity that is already occurring. Neither alone forecasts the future. Predictive analysis uses past data to estimate which newly released vulnerabilities are likely to cause compromise, allowing teams with limited remediation capacity to intervene earlier and concentrate work where it can reduce more risk.

Q: What is the difference between CVSS severity and vulnerability risk?

CVSS provides a point-in-time assessment of the software flaw itself, using categories such as access complexity, authentication method, and confidentiality impact. That information is useful for understanding potential impact, but it does not directly establish whether attackers will exploit the flaw. Vulnerability risk also depends on exploitation likelihood, so severity should not be treated as a complete prioritization measure.

Q: When are vulnerabilities most likely to be exploited after release?

The presented data indicates that exploitation probability increases during the days following publication and reaches an inflection point around 40–60 days. However, exploitation can occur earlier because the probability is non-zero on release day and reaches about 25 percent by day 10 in the scoped data. The operational objective is therefore to identify high-priority vulnerabilities as soon after release as possible.

Q: How effective is fixing every CVSS 10 vulnerability?

Across the cited data set of 1.4 billion live vulnerability instances, remediating all CVSS 10 vulnerabilities would address about 11.5 percent of vulnerabilities associated with successful exploitation. That is roughly twice the approximately six percent result expected from random selection. However, CVSS 10 vulnerabilities represent 28 percent of the vulnerabilities, so the strategy requires considerable remediation capacity for its achieved risk reduction.

Q: Why are Exploit-DB and Metasploit useful for prioritization?

An entry in Exploit-DB indicates proof-of-concept exploit code, while an entry in Metasploit represents a weaponized form of exploit code in the speaker's comparison. When a vulnerability appears in both, the remediation hit rate rises to 36 percent. This group accounts for about 15 percent of vulnerabilities, so it produces a stronger relationship between remediation work and successfully exploited vulnerabilities than CVSS 10 alone.

Q: How should teams evaluate a vulnerability remediation strategy?

Teams should evaluate a strategy using three connected measures: the amount of work required, the amount of risk reduced, and the effectiveness of each remediation action. A rule that selects many vulnerabilities can consume capacity without proportionate benefit. A better comparison asks what share of the environment must be remediated and how often those fixes address vulnerabilities associated with successful exploitation.

Q: Which vulnerabilities should teams fix first in a large backlog?

For an established backlog, teams should begin with old, stable, weaponized vulnerabilities. Attackers already know that these vulnerabilities work, scan the internet for them, and use them in prevalent non-targeted attacks. This approach addresses accumulated security debt, while predictive models are needed for the separate problem of prioritizing newly released vulnerabilities before the usual exploitation evidence becomes available.

Summary & Key Takeaways

• Vulnerability management commonly relies on retrospective severity assessments and real-time threat intelligence. CVSS describes characteristics and potential impact at a particular point, while event feeds report exploitation activity already observed. Predictive analysis goes further by using past data to generate new estimates about which vulnerabilities attackers may exploit.

• The available data shows that exploitation likelihood rises after publication, with an inflection point around 40–60 days. Risk is still present immediately, and the probability reaches about 25 percent by day 10 among the scoped vulnerabilities. Teams therefore benefit from identifying dangerous vulnerabilities as close to release as possible.

• Remediation strategies should be judged by required effort, total risk reduction, and effectiveness per action. Fixing every CVSS 10 vulnerability requires substantial capacity and reaches about 11.5 percent of vulnerabilities associated with successful exploitation. Combining Exploit-DB and Metasploit indicators raises that figure to 36 percent while covering fewer vulnerabilities.