How to Build Security Metrics Executives Can Use

TL;DR

Build a security metrics program around existing operational data, intuitive navigation, and executive self-service reporting. ADP’s approach connected risks, incidents, control failures, technologies, projects, resources, and costs, reducing monthly executive reports by more than 70% while helping leadership make security decisions more frequently and refine an initially broad server risk estimate from 10% to about 1%.

Transcript

Time for us to get started here. Uh, just wanted to introduce ourselves, James Lougeeville, Ma- Marta Palenkas from, uh, ADP. Uh, we're part of the execution assurance team within our global security organization. Uh, we're gonna... here to talk, talk metrics, and it's pretty, uh, surprising to see that this is a replay, and there's a lot of you gu... Read More

Key Insights

• Executive self-service is the central reporting goal because leaders often need answers within days, especially during budget planning, regulatory responses, or reactions to breaches. An accessible platform allows them to investigate questions directly instead of waiting for analysts to assemble one-off reports manually.
• Security data is often fragmented across numerous systems of record, requiring substantial manual processing before it becomes meaningful. When sources are disjointed or contradictory, reporting becomes inconsistent and decision-makers may lose confidence in both the underlying data and the resulting deliverables.
• The battlefield view connects risks, incidents, failed controls, technologies, deployment, maturity, projects, resources, and costs. This structure lets a CISO navigate between related concepts, understand organizational conditions, and identify which operational or investment levers may need adjustment.
• Existing data is the sustainable foundation for security reporting because creating information solely to populate reports adds work that becomes difficult to maintain. ADP chose to leverage information already available in its environment while gradually expanding the range of services and data sources included.
• Intuitive navigation is essential because executives need to move quickly from one security concept to another and construct their own decision narrative. The reporting experience must make relationships visible without requiring leadership to depend on analysts for every follow-up question.
• ADP’s metrics program reduced monthly executive reports by more than 70% after about 18 months of work involving roughly one and a half full-time employees. The saved effort created capacity to investigate additional metrics, information sources, and reporting opportunities.
• Executive decision frequency increased as reporting coverage expanded. During the prior fiscal year, decisions occurred in about one of every three monthly security meetings, while the current fiscal year saw decisions in nearly every meeting, sometimes with multiple decisions during a month.
• Detailed analysis can materially refine a broad risk estimate. A concern initially appeared to affect about 10% of tens of thousands of servers, but consolidating information by factors such as location, application, and operating system reduced the relevant estimate to about 1%.

Questions & Answers

Q: How do you build a security metrics program for executives?

Build the program around leadership questions, executive self-service, intuitive navigation, and data that already exists within the organization. Connect information about risks, incidents, control failures, technologies, deployment, maturity, projects, resources, and costs. This allows executives to investigate related issues directly, understand the broader security environment, and make decisions without waiting for analysts to create a separate report for every request.

Q: Why should security reporting use existing operational data?

Existing operational data provides a more sustainable basis for reporting than information generated solely for presentation purposes. ADP had experienced the difficulty of creating data just to support reports, especially across numerous security topics. Reusing available sources reduces additional collection work, supports broader coverage, and lets the metrics program expand gradually as more services and systems of record become connected.

Q: What is executive self-service security reporting?

Executive self-service security reporting is a platform that allows leaders to explore security information and answer their own questions. Instead of relying on analysts to manually combine fragmented sources under short deadlines, executives can navigate among risks, incidents, controls, technologies, projects, resources, maturity, and costs. The objective is to provide a broad battlefield view that supports timely operational and investment decisions.

Q: How can security metrics connect risk to business investments?

Security metrics can connect investments to risk by linking a risk issue with any resulting incident, the related control failure, the supporting technology, and that technology’s deployment and maturity. Reporting can then connect the technology to projects, assigned resources, maintenance spending, and improvement costs. This chain helps leadership understand how operational conditions, project activity, and spending relate to security outcomes.

Q: How did ADP reduce its executive security reporting workload?

ADP implemented executive self-service reporting and a structured methodology for capturing metrics. After about 18 months, with roughly one and a half full-time employees working on the solution, the organization had reduced monthly executive reports by more than 70%. That reduction gave the team additional capacity to investigate more metrics, integrate further information, and expand reporting beyond the services already covered.

Q: What results did ADP achieve with its security metrics program?

The program covered 12 core security services and drew from six data sources at the point described. It reduced monthly executive reports by more than 70% and coincided with more frequent leadership decisions. Decisions had occurred in about one of every three monthly meetings during the prior fiscal year, but were occurring in nearly every meeting during the current fiscal year.

Q: How can detailed metrics refine a security risk estimate?

Detailed metrics refine a risk estimate by segmenting and consolidating the underlying information rather than relying on a broad initial percentage. In ADP’s example, a concern appeared to affect about 10% of tens of thousands of servers. Analysis across dimensions such as location, application, and operating system narrowed the relevant estimate to about 1%, giving decision-makers a more precise view.

Q: How should security metrics be presented to executives?

Security metrics should be presented through concise visualizations and an intuitive interface that makes important conditions immediately identifiable. Executives need to move quickly among connected topics and build a coherent story from the data. Reporting should therefore emphasize clear relationships and actionable information instead of relying only on long explanations or isolated numbers that require analysts to interpret them.

Summary & Key Takeaways

• ADP developed an executive self-service reporting platform to replace slow, fragmented responses to leadership questions. The platform lets decision-makers explore connections among risks, incidents, control failures, technologies, projects, resources, maturity, and costs, creating a broad battlefield view of the global security organization and supporting rapid adjustments when requirements or threats change.

• The program was built around two foundational principles: reporting must be intuitive enough for executives to navigate independently, and it must use data already present in the environment. Generating new data solely for reporting was considered unsustainable, particularly because the organization needed to cover many security services, technologies, locations, and data sources.

• After about 18 months and roughly one and a half full-time employees of effort, the program covered 12 core security services and six data sources. It reduced monthly executive reports by more than 70%, increased the frequency of leadership decisions, and helped narrow one server-related risk estimate from 10% to about 1%.