Why Vulnerability Counts Are the Wrong Security Metric

Ask a CISO how their security programme is performing and you'll often get an answer that sounds like this: "We closed 3,400 vulnerabilities last quarter — up 22% on Q2."

It sounds like progress. It might even be presented on a slide with a green upward arrow. But it tells you almost nothing about whether the organisation is actually safer.

The number of vulnerabilities closed is a throughput metric. It measures activity, not risk reduction. And the difference matters enormously — because activity and impact are not the same thing.

The problem with counting findings

The typical enterprise runs somewhere between five and fifteen security scanners. Each one generates findings. Add them all up and you might be looking at tens of thousands of open vulnerabilities at any given moment.

What do most security programmes do with this? They set a target — close X% of critical findings within Y days — and measure performance against it.

The problem is that not all critical findings are equally critical. A critical CVE on an internet-facing customer application that processes financial data is categorically different from the same CVE on a development sandbox with no external access and no sensitive data. Both show up as "Critical" in the scanner. Only one represents serious business risk.

When you optimise for count-based metrics, remediation teams — developers, IT, infrastructure — naturally gravitate toward whichever findings are easiest to close. That usually means patching systems that are simple to patch, not systems that are most exposed or most business-critical.

What makes a vulnerability actually matter?

The risk of any given vulnerability is a function of at least four things that a scanner score doesn't capture:

1. Exploitability in the wild. Is there an active exploit? Is it being used by threat actors right now? A CVE with a CVSS score of 9.8 that has no known exploit and no threat actor interest is meaningfully less urgent than a CVE with a CVSS of 7.2 that appears in current ransomware campaigns.
2. Asset exposure. Is the affected system reachable from the internet? From the internal network? Isolated in a development environment? Exposure dramatically changes the attack surface.
3. Business context. What does this system do? Does it process customer data, financial transactions, regulated information? Who owns it? What business service depends on it? A vulnerability in a payment processing service has a completely different risk profile than the same vulnerability in an internal wiki.
4. Compensating controls. Is there a WAF in front of this? Is the system behind network segmentation? Are there detective controls that would catch exploitation attempts? These factors change the effective risk even when the raw vulnerability exists.

None of these appear in a scanner output. They have to be enriched — connected to asset inventory, business context, threat intelligence, and network topology.

What to measure instead

Moving from activity metrics to impact metrics requires a shift in what you track:

Risk reduction over time

Rather than counting vulnerabilities closed, track the reduction in your organisation's overall risk exposure — weighted by exploitability, asset criticality, and business impact. This tells you whether your programme is making the organisation meaningfully safer, not just busier.

Coverage of highest-risk findings

What percentage of your top-risk findings — those with active exploits on business-critical assets — are remediated within SLA? This is a much tighter metric than "critical findings closed," and it focuses the organisation on what actually matters.

Mean time to remediate by risk tier

MTTR is useful when broken down by risk tier rather than reported as a single average. A 74-day average MTTR might hide a 12-day MTTR for your highest-risk findings and a 200-day MTTR for medium-risk ones. That breakdown changes the conversation entirely.

Risk accepted vs. risk resolved

Tracking exceptions and risk acceptances tells you what your organisation has consciously decided to live with. When exceptions are tracked, patterns emerge — teams that are consistently accepting high-risk findings, asset owners who are systematically underprioritising remediation, or classes of vulnerability that are being waved through without adequate scrutiny.

Security investment effectiveness

This is the hardest metric but the most valuable for leadership conversations: given what we've spent on security tools and headcount, has our risk exposure decreased? Are we getting faster? Are we addressing the risks that matter most?

Why this shift is hard

The reason most organisations stick with count-based metrics isn't ignorance — it's that risk-based metrics require data that most organisations don't have readily available.

To know that a vulnerability on System A is three times more business-critical than the same vulnerability on System B, you need to know:

• What System A does and what business service it supports
• Who owns it and who is responsible for patching it
• What data it holds and what compliance frameworks apply to it
• Whether it's internet-facing or internally accessible only
• What the current threat actor activity is for that specific CVE

That context exists — in CMDBs, asset management systems, business continuity plans, and threat intelligence feeds — but it's fragmented across systems that don't talk to each other. The scanner sees the vulnerability. It doesn't see the business.

The path forward

Organisations that make the transition from activity metrics to risk-based impact metrics typically go through three stages:

Stage 1 — Consolidation. Bring all findings from all scanners into one place so you're not managing five separate queues with five separate metrics. Deduplication alone often reveals that 30–40% of your "open findings" are the same vulnerability reported by multiple tools.

Stage 2 — Contextualisation. Enrich every finding with the information scanners can't provide: asset ownership, business criticality, network exposure, and threat intelligence. This is the step that makes prioritisation possible.

Stage 3 — Measurement. With consolidated, contextualised data, you can track the metrics that actually reflect security outcomes — risk reduction, coverage of high-risk findings, MTTR by tier, and programme effectiveness over time.

None of this happens overnight. But the direction of travel is clear: security programmes that measure outcomes rather than activity consistently demonstrate more value, make better use of remediation resources, and reduce the business risk that security exists to manage.

The number of vulnerabilities you've closed is not the story. Whether your organisation is safer than it was six months ago — that's the story.