The ability to measure performance is at the heart of any effective management, and performance measurement is essential to allocate resources, select operations, and reward personnel. To assess this performance, executives use massive amounts of data and metrics like return on investment to determine which products increase their company’s value, they should use return on security investment also.

However, measuring cybersecurity performance is challenging. Traditional performance metrics such as revenue or cost are irrelevant, and analogs to market and credit risk metrics like value at risk do not exist for cybersecurity. Measuring cybersecurity incidents might lead people to believe that security teams protect the organization, although they actually can’t detect ongoing attacks.

Analysts have been trying to identify possible risks and damage and the probable occurrence of cyberattacks to not miss out on the necessary security investments. However, these risks and probabilities are difficult to account for, and some companies only make drastic changes in their strategies once the first attack has occurred.

Metrics exist but, whether executives look at the Return on investment (ROI) or the Return on security investment (ROSI) to try to assess the quality of their investment, we think that the main challenge is understanding that cybersecurity investment is about linking an actual expense with an unforeseeable threat.

Why assessing the quality of a cybersecurity investment is hard according to traditional metrics

Quantifying the returns of cyber security is challenging, given the fact that it is a preventive measure. Cybersecurity neither impacts the revenues directly nor does it provide immediate payback. Thus, traditional metrics are not reflecting the investments.

Cyberattacks have specifics that are hard to put a price on.

The growing awareness has given rise to significant increases in the budgets allocated to information security departments.

However, it is often complicated for a Chief information security officer (CISO) to demonstrate the benefit of security expenses since cost avoidance remains the prevailing paradigm. CISOs have to put a value on what hasn’t happened yet.

In addition, the cybersecurity environment is dynamic. Threats escalate fast; new technologies and operational practices appear every day. This environment makes it challenging for CISO, already struggling to control their tool stack, stay afoot and determine whether their security architecture is sound or the need to add some tools or reduce their stack.

More, assessing the costs of a cyberattack may prove very complex as it varies according to the type of breach, its length, or the sensitivity of data stolen. (metrics on cost of data breaches). Along with this financial impact, possible regulatory penalties should be considered. For example, companies facing a data breach could be fined under the GDPR – up to 4% of the worldwide turnover – if convicted of being negligent in protecting these data.

Further, sometimes, the assessment of incidents fluctuates considerably. For instance, what is the cost of one-hour unavailability of an e-commerce site during the sales period following a DDoS attack? Cyberattacks have the particularity to have the potential to impact the availability, reputation, and confidence, which are intangible, hard to price.

The Return on Investment is the traditional primary metric in terms of assessing the quality of an investment.

Traditionally, the Return on Investment (ROI) is calculated based on returns on cost and measures the amount of return on a particular current investment cost (IC) relative to the initial investment’s price (IP). It is calculated like so:

Therefore, assessing ROI works for investments that yield positive results, such as cost savings or revenue enhancements. ROI is a metric used to evaluate the efficiency or profitability of an asset or compare the efficiency of several different investments. It’s not designed to determine how much loss the organization avoided with its initial investment.

How can companies properly evaluate their investment in cybersecurity?

Executives have to account for actual expenditures that aren’t readily apparent against an unforeseeable threat. They then need to shift from a paradigm focused on cost minimization to one that encourages investments. Both quantitative and qualitative ways are possible to evaluate the investment correctly; organizations should mix both.

Companies should use the return on security investment

Cyber security is a pre-emptive investment. Its goal is not about increasing the value of the initial investment but avoiding its depreciation due to hostile actions. As such, measuring the return the classical way sounds irrelevant. Any assessment of a cybersecurity investment should be based on how much loss the organization could avoid due to the acquisition, not positive value added to the initial cost.

According to this specificity, the European Union Agency for Cybersecurity issued a methodology more suited to calculate a company’s returns when investing in cybersecurity solutions. As such, estimating the Return on Security Investments (ROSI) entails identifying expected losses prevented and the rate of risk mitigation. The equation is written as follows:

Here, the Annualized Loss Expectancy (ALE) is the estimated amount of money that a company will lose in a single security incident (single loss expectancy, SLE) multiplied by the estimated occurrence of a threat within a year (annualized rate of occurrence, ARO). Usually, CISOs would estimate ALE in regards to the successful breaches that happened in the year prior to the adoption of a new solution.

The Mitigation Ratio (MR) is an approximate number determined by assessing the number of mitigated risks based on an algorithm established. For instance, if a company invests in a solution expected to reduce the current data security risk by 90%, then MR equals 90%. Although it is unrealistic to assume any solution to be a 100% guarantee, CISOs can assess reasonable expectations when looking at a given solution.

Finally, the cost of Solution (CS) is the only independent index in this equation, and it includes the costs associated with solution purchase, implementation, and maintenance. CISOs have to make sure that they consider all costs, not just the hardware or software one-time cost. This includes maintenance, subscription to a Threat Management service, the time and cost to train staff, and the time needed to configure and implement the solution.

You can add other qualitative and quantitative metrics to enrich and enhance your ROSI

After dissecting that equation, it’s easy to see that some of the parts are hard to estimate accurately. Thus, CISOs should try to enrich their ROSI metrics with other methods to further precise the calculus. It goes by comparing with peers or planning red team exercises. Eventually, one way to enhance ROSI is choosing automation and orchestration with the help of a Security orchestration, automation, and response (SOAR).

Compare your organization with your peers and with public resources.
Finding accurate estimations for SLE and ARO is complicated. For Single Loss Expectancy, CISOs can look at internal sources such as figures for the cost of production line downtime, or office productivity, or actual losses after an attack. Still, to assess the exact cost, CISOs would need to be able to price the reputational and confidence damages. CISOs should keep in mind that there always is intangible damages occurring after a cyberattack.

However, as for financial metrics besides internal sources, there are also external ones a CISO can look into. IBM and the Ponemon Institute, or Verizon, put out annual Data breach reports that can give pieces of information on losses in various industries on an international scale. The same goes for the ARO; any internal history the organization may have will be useful, but there are also a variety of reports on the subject that may be of help.

Also, more generally, comparing security budgets with those of other organizations in the industry is an excellent way to gauge the effectiveness of security investment. To that end, a company could get the help of a consultancy firm.

Evaluate the readiness of your organization to address incidents
One great way to assess the quality of security investment is to carry out security simulations. Separate teams have the task of infiltrating and defending a specific infrastructure. It is a good way of staying in touch with current security scenarios and identifying weak links. It also helps test the effectiveness and opportunity of a tool, check the level of security awareness in the organization, and measure the performance of each IT team member.

Performing simulations on a regular basis can serve as an excellent practical metric of how cyber security investments are affecting the organization.

Implement automation and orchestration to increase your ROSI
Today Security orchestration, automation, response (SOAR) solutions provide some of the highest cost savings opportunities with AI and automation. It helps CISOs resolve some of security professionals’ burdens, such as the sheer volume and sophistication of attacks coming in, the dwell time of breaches, the high rates of false positives, the time and resources required to respond, and the cyber skills shortage.

Further, automating and streamlining incident management reduces the time to detect and respond, besides increasing the detection rate and lowering the possibilities of false positives.

Eventually, orchestration between each tool, enhancing their connectivity and visibility, is a way for CISOs to continuously assess the opportunity of the current composition of the security stack, keep all the tools up to date, and limit the percentage of unused tools although allowing working staff to use the tools they want.

Conclusion

An investment in cybersecurity cannot be assessed with the traditional metrics used as the return on investment. Such an investment is made to avoid costs, not to deliver positive results. Using the return on security investment helps overcome that issue.

However, this approach needs to be complemented with other measures, such as comparisons and tests. Also, we are convinced that SOARs are solutions that have to be implemented because they drastically reduce the costs and enhance the processes simultaneously.