The K in Cyber Security KPIs

The stakes involved in flying are higher than in cyber security. No one should disagree with that statement. 

With all of those high potential stakes, think about the airline key performance indicators (KPIs) that matter to you as a passenger when flying.
  
That your plane arrives at the destination.

That the plane arrives on time.
That emergency procedures are in place.
That your luggage arrives with your flight.

Each of the above is an easily digestible end state, a business outcome. Simple questions that mask the “white space” or complex activities that comprise each of those outcomes.

The Potential Downside of Cyber Metrics

You always get what you measure.That phrase or some paraphrase of it is usually meant to justify some positive change. 

For example, we were doing such-and-such activity before with poor results and then we started measuring and got better results.

This is a common statement and standard narrative in any cyber security program.

When the stakes are high, we want to be sure to turn the box green, finish within the right time boundary, or get results above the right percentage.

The Cyber Security Metric Menagerie

Anything can be measured and turned into a metric.

Some of these metrics are meaningful. Most aren’t. The most meaningful metrics measure things that are related to your business goals.

The Vulnerability Management Fantasy

My team doesn’t patch. We govern patching.

The patch incoming rate is dizzying. We have OS patches, application patches, firmware patches. All need to be tested and deployed.  It’s a never ending race with seemingly no finish line of three of our teams against one potential event. 

The vulnerability risk score can become what’s important and what’s reported. It’s a metric that easily generated but often needs explanation to execs. It is also  really hard to compare your score against others in your industry or company size. 

Only vendors can fix the risk score problem

Develop Security Metrics That Also Are Your Remedy Negotiation

I have a few deceptively simple rules about metrics

• We should know the specific question each metric answers
• We should know why each question is important to the program
• We should know in advance what levers to pull if the metric goes off track

The most effective metrics a security program can report to leadership would be clearly linked to the larger organization's strategy.  Anyone would see these as important and make the linkage/value of the cyber security team to the company an obvious one.  

Determining the important things for your cyber security program takes thinking, input, and agreement from other execs. Unfortunately, these linkages probably aren’t represented in the default metrics that come out of the box from your security tools.  Presenting inputs and outputs that are critical to your security program ideally give your leadership team a “check engine light” and an action plan long before there is a real issue that impacts the company.

One of my team's best output metrics right now is the number of revenue producing staff hours lost to security incidents. It’s an ambitious output with a lot of white space and multiple inputs built into it that keep the team focused on executing well in the weeds. It also has an obvious link to a company that wants to sell things.  The executives already know that this metric won’t be zero forever. If the metric begins to move more than a small number of hours, it means that either the security threat landscape has shifted in some way or we are lacking a key control that is impacting revenue. Perhaps it means both. A significant movement in this metric means that I’ll likely have to spend money. Perhaps even unplanned money. The execs already know this too. If I know the average cost of a staff hour, the metric quickly approximates the revenue impact when I present the cost of the new control to compensate for a change in threats. That metric makes what used to be a hard conversation much easier.