How to Develop Metrics That Guide Decisions

TL;DR

Start with a business objective and define what decisions or behaviors a metric should influence before selecting data to measure. The Goal Question Indicator Metric process derives meaningful metrics from organizational objectives, frames them in language senior leaders understand, and helps test whether current reporting provides useful business intelligence or merely produces numbers without context.

Transcript

Okay, good. So we decided we were gonna sit down instead of jumping up and down every time it was, you know, for one of us to speak. So thank you guys for being here. I really appreciate it. It's Friday morning. Um, we're teeing up the keynote. So, um, I'm Lisa Young, and I know it says Vice President, Service Delivery, but that sounds so boring th... Read More

Key Insights

• Measurement objectives are necessary before starting a measurement program because organizations need to know which performance objectives their security programs are expected to meet. If those objectives have not been defined or codified, that foundational work must occur before useful metrics can be derived.
• A measure is a direct observation or quantity, such as eating two eggs or recording a temperature of 53 degrees in San Francisco. A metric goes further by placing information in context and producing business intelligence that can support data-driven decisions.
• The purpose of a useful metric is to inform a decision, change a behavior, or improve a function. Before collecting or reporting it, practitioners should ask what decision it would influence, what behavior it would alter, and what observable improvement would look like.
• A large collection of favorable numbers is not necessarily a metrics program. The CIO who received ten pages of green metrics while spending every day solving problems demonstrated how reporting can fail when its indicators do not reflect operational reality or answer meaningful questions.
• Business-focused language makes technical reporting more relevant to senior leaders. Saying that a server is down provides limited context, while connecting that outage to an inability to accept customer payments identifies a direct consequence for revenue and changes the resulting management dialogue.
• The Goal Question Indicator Metric process is a defined and repeatable method for deriving metrics from objectives already present in an organization. Its purpose is not to prescribe specific technical measures, but to connect organizational needs with questions, indicators, and decision-supporting information.
• The GQIM approach was adapted from the Goal Question Metric method developed for software measurement in the NASA Goddard Space Program. Carnegie Mellon practitioners used the established method, added a separate indicator element, and applied it to security, resilience, risk management, and strategic objectives.
• Existing metrics should be tested for utility by running them through the GQIM technique. The evaluation should determine whether a metric answers a business question, supports an objective, informs a decision, or changes behavior, rather than assuming that continued reporting proves its usefulness.

Questions & Answers

Q: How do you develop useful cybersecurity metrics?

Begin by identifying the business or program objective that creates the need for cybersecurity or resilience. Then determine what you want to know, which questions must be answered, what decisions the resulting information should inform, and which behaviors should change. The GQIM process provides a defined, repeatable way to derive indicators and measures from those existing organizational objectives.

Q: What is the Goal Question Indicator Metric process?

The Goal Question Indicator Metric process, abbreviated GQIM, is a method for deriving useful metrics from organizational objectives. It connects goals to the questions decision-makers need answered, then identifies indicators and supporting measures. The approach is intended to produce contextual business intelligence rather than disconnected numbers, and it can be applied across cybersecurity, resilience, risk management, and strategic programs.

Q: Why should metrics start with business objectives?

Metrics should start with business objectives because measurement only has value when it supports something the organization is trying to accomplish. A defined objective clarifies what the security program should achieve, what information leaders need, and what improvement should look like. Without that foundation, teams may collect many numbers that neither guide decisions nor reveal whether performance objectives are being met.

Q: What is the difference between a measure and a metric?

A measure is a direct quantity or observation, such as two eggs eaten for breakfast or a temperature of 53 degrees in San Francisco. A metric places one or more pieces of information into context so they provide business intelligence. That context enables the information to inform decisions, support comparisons, reveal trends, or influence behavior rather than merely report a number.

Q: How can security teams communicate metrics to business leaders?

Security teams can communicate more effectively by translating technical conditions into consequences the business recognizes. Reporting that a server is down may not explain why an executive should care. Reporting that the outage prevents the organization from accepting customer payments connects the same technical event to revenue. Reusing the business language expressed by senior leaders can also improve program visibility.

Q: How can an organization tell whether a metric is useful?

A metric is useful when it answers a meaningful question, informs a decision, changes a behavior, or helps improve a function. Practitioners should ask why the metric exists, what action would follow from it, and what better performance would look like. If it cannot support a business objective or explain relevant conditions, its continued production should be challenged.

Q: Why can green dashboards still fail to reflect reality?

Green dashboards can fail when reported indicators are disconnected from the problems leaders actually face. One CIO received ten pages of metrics showing green results while spending entire days resolving issues and attending meetings where conditions were not green. The mismatch showed that extensive reporting and favorable status labels do not guarantee an accurate or useful measurement program.

Q: When should teams review their existing security metrics?

Teams should review existing metrics when assessing the utility of their measurement program and can begin with one or more reports they already produce. Running each metric through the GQIM technique helps determine whether it supports a business objective, answers a relevant question, informs a decision, or encourages improvement. This assessment is presented as a practical first use of the method.

Summary & Key Takeaways

• Useful measurement begins with clear objectives, not with a list of available numbers. Organizations should determine what they want to know, which decisions the information should support, what behaviors should change, and what improvement would look like. These questions establish whether a proposed metric has practical value for the business.

• The Goal Question Indicator Metric process is a defined, repeatable derivation method adapted from the Goal Question Metric approach used for software measurement. It can be applied to cybersecurity, resilience, risk management, strategic objectives, program measurements, and hygiene measurements by connecting organizational goals with questions, indicators, and supporting measures.

• Security reporting becomes more useful when technical conditions are translated into business consequences. Reporting only that a server is unavailable may not engage senior executives, while explaining that the outage prevents customer payments and revenue collection changes the discussion. Business language also improves program visibility and helps demonstrate value or justify investments.