Risk has always been tricky to assess but essential to any business. As the saying goes, “you can’t manage what you can’t measure”.. Metrics are diverse. Some are complex to evaluate; others focus on things hard to assess by essence. Any program needs metrics to measure its success and help make choices.
As in any other program, cybersecurity metrics are diverse, and having detailed risk exposure data is challenging. Even today, Chief information security officers (CISO) are often not confident that they are using adequate cybersecurity metrics when monitoring their activities.
However, cybersecurity differs from other fields in the constant development of the threat landscape and the technologies used by attackers and defenders. Cybersecurity metrics are thus of dire importance to stick with the evolution.
As the attacks grow and their consequences, the interest in cyber resilience also increases at shareholders, regulatory, and board levels. A CISO needs to provide more and more actionable cybersecurity metrics to demonstrate the opportunity of their growing costs to inexperienced decision-makers.
But measurements are also crucial at a technical level. Security operations centers (SOC) use cybersecurity metrics to determine the effectiveness of their operations and identify ways of progress on their incident management programs. In short, metrics are a driver for continuous improvement.
So the choice of cybersecurity metrics is crucial on these two points: improvement and justification. Below we’ll define:
- What makes good cybersecurity metrics
- Which cybersecurity metrics should you use
- A look at a comprehensive list
What makes good cybersecurity metrics
Defining a set of good cybersecurity metrics is in itself a challenging task. To assess a good stack, people in charge need to have an exhaustive overview of the company: business, technology, specific risks, or what information needs to be protected and why.
Also, metrics differ; some target the macro or forward level, whereas others focus on the micro or backward level. In the end, we could even dare to say that there are as many metrics as people in charge of defining them.
Among all of these metrics, it’s easy to get lost and lose the meaning behind the numbers. Thus, when undertaking the work of assessing a set of metrics for their organizations, people often refer to the “SMART” structure. It stands for:
- Specific: designed to the area measured
- Measurable: accurate and complete
- Actionable: simple to understand and to take action on
- Relevant: what's necessary
- Timely: available when needed
Metrics defined according to this design bring clarity, structure, and measurability. It turns a number into something much more apprehensible.
But behold! SMART is good but not enough.
In the end, a good metric doesn’t come alone. A number makes sense only when further data accompany it. Metrics can only come after you decide on Goals. What are you chasing? What are your objectives? Goals give you direction.
Also, it would be best if you defined what’s acceptable or not, a target range for each measurement. You don’t need to chase 100% everywhere. On the opposite, actually. The five nine rules help you better allocate your resources. The point is to keep in mind that no system is infallible. Every one of them ends up breached at some point. Trying to fill the gap between 99.999% and 100% would cost you an exponential amount of time and people better spent somewhere else.
This is why you have to define lower and upper bounds for each metric; an acceptable range that presides the allocation of efforts of your teams.
To sum up, measurements are part of a broader plan of governance. Each has to go by specific goals, action plans, and targets. Even SMART ones cannot come alone, which is often the case.
- Define Goals
- Define Action plan
- Asses metrics
- Set targets that will be used to monitor metrics regularly
Comparison is one of the keys to assessing your posture strength. Metrics deliver their best when compared. Peers and competitors are facing the same threats like you. They’re trying to enact the same policies. Thus, by comparing with peers and the global average, you’re determining whether you’re over or underperforming.
Emphasis on the understandability of the cybersecurity metrics
We cannot stress enough the understandability of the cybersecurity metrics you’ll use because of the exponential growth of cyberattacks these last two years. The CISO saw its role gaining immense traction before the boards. Beyond emergencies, they increasingly see cybersecurity as a necessary collaboration, embedded in deeper ties with other departments.
This new relationship requires the CISO to have a more thorough understanding of C-level concerns and priorities and how these are translating into information and infrastructure security. The board must, in turn, acknowledge how vital cyber risk has become to business outcomes and consider it when it comes to taking responsibility.
The CISO’s role is to influence and inform. To that end, you need to communicate with simple and actionable data. That said, you will want to choose some metrics that are clear to anyone, even non-technical stakeholders.
And remember that one universal metric is cost. We’ve discussed this in an earlier article: cybersecurity’s goal is to save your company money. Knowingly, by comparing to peers your results or on a year-to-year basis, you should be able to synthesize your point in actionable vital data.
With that being said, let’s dive into strictly speaking metrics.
Which cybersecurity metrics your company should use
In Enterprise Risk Management, specialists rely on Key Performance Indicators (KPI) and Key Risk Indicators (KRI), sometimes one instead of another or both under one name. These metrics are perfectly suitable for cybersecurity but let’s take a minute to redefine them.
KPI is a measure of performance. It’s, by nature, a backward-looking metric, coming at a higher-level overview. It’s not designed to provide early warnings but to analyze trends.
KRI is a measure of risk exposure. At the opposite of KPI, it’s forward-looking, at a lower and more technical level.
They’re two different metrics but complementary because KRI comes before KPI and informs them.
KPIs and KRI aren’t the same. KRI helps to quantify risks, while KPI helps to measure business performance.
Bear in mind that, as said above, to align performance management and risk management and be as understandable as possible, you should link each KRI to a KPI. KPI has long played an essential role in performance management.
A comprehensive Security governance policy should be hierarchized into:
- Goals
- KPI
- KRI
- Targets
Here’s a pick at our list of cybersecurity metrics
Let’s look at the selected KPI and KRI with that in mind. Again, you should accompany these with pre-established goals as to the objectives you want to achieve and, more specifically, targets for each KPI’s and KRI’s.
Also, KPI and KRI need to be coherent. For instance, virus monitoring participate in clearing up the number of incidents reported. Remember that KRI and KPI are complementary.
Key Risk Indicators (KRI)
- Level of Readiness: Number of devices on your corporate network that are patched and up to date
- Unidentified Devices on network: Malware can find their way into your network with employees bringing their personal devices
- Vendor Security Rating: cybersecurity metrics must follow the threat landscape that goes beyond your company’s limits. You have to assess your vendor risk management and a third-party risk management framework to complete your metrics
- Third parties dependencies: as the Log4Shell exploit proved, you need to be aware of all the dependencies you’re relying on to assess your risk exposure properly
- Patching Cadence: the time needed to implement application security patches or mitigate high-risk CVE-listed vulnerabilities? Cybercriminals use threat intelligence databases to benefit from the delta between patch releases and effective implementation. WannaCry exploited EternalBlue to make its way into computers. This zero-day vulnerability was quickly patched, but many companies still got breached due to a lag in a patching policy
- Antivirus monitoring: periodicity of antivirus scans on apps (email clients, web browsers, and instant messaging software)
- Access Management: how segmented your network is. Many breaches are due to insider threats, such as negligent users with high privileges
- Backup Cadence: on what periodicity you’re making backups of your data
- Number of known vulnerabilities on externally facing and internal systems: an exhaustive view of your assets classified by their risk exposure
Key Performance Indicators (KPI)
- Number of alerts: how many alerts have been received daily
- Number of false positives alerts: how many false positive alerts are SOC analysts encountering daily
- Number of reported incidents: how many reported incidents within a specific timeline
- Mean Time to Detect (MTTD): number of days breaches go unnoticed
- Mean Time to Remediate (MTTR): number of days between detection and remediation
- Mean Time to Contain (MTTC): number of days until identified attack vectors are barred across endpoints compromised
- Cost induced by incidents: on a monthly and annual basis, how much did breaches cost. Costs include investigation costs, productivity loss, downtime, regulatory sanctions, and data loss
- Analyst workload: the number of events that your analysts have to deal with daily
- Phishing attack success: following phishing and spear-phishing campaigns, the percentage of positives
- Cybersecurity awareness training: the recurrence, comprehensiveness, in terms of employees concerned (C-suites included), and results
The last metric would be a general recommendation to compare these cybersecurity metrics with your peers and global averages. As said above, comparisons are easily understandable and appealing for board talks.
