Applying NPV and ROI to Security Investment Decisions

Sunday, October 10, 2010

Kurt Aubuchon

972cda1e62b72640cb7ac702714a115f

Abstract

Often, the decision of whether or not to implement a security countermeasure is ultimately based on a vague appeal to "best practices", or on a gut feeling that the cost of the countermeasure is justified by the risk of an exposure.  

In this article, I propose a model, based on Net Present Value, Return on Investment, and Monte Carlo simulations, that provides a quantitative framework for these decisions.  A sample analytical tool is also provided.

Introduction

Justifying the costs of security expenditures is a persistent challenge for the Information Security Manager. 

The often significant implementation and maintenance costs of a countermeasure, coupled with the difficulty of quantifying the costs of a breach, make traditional cost-benefit analyses problematic at best, and simple guesswork at worst. 

While information security literature addresses such concepts as Single Loss Expectancy (SLE), Annualized Loss Expectancy (ALE), and Annualized Rate of Occurrence (ARO), there are few good models available to translate these concepts into the kind of business analyses that are persuasive to operational managers. 

In addition, the models that are available often fail to adequately take into account the probability and timing of a security breach.  An extravagant SLE will fail to persuade senior executives who perceive a vanishingly low probability that such an event will occur.

To be successful, the Information Security Manager (ISM) must be able to make a case for security investments in terms that are persuasive to business managers, not necessarily to other information security professionals. 

Senior management will be unmoved by appeals to IT best practices; they will become inured to frightening tales of companies who paid the price for lax security practices; and they will lose confidence in a security manager who supports proposals with little more than instinct. 

The ISM must be able to draw on traditional analytical tools such as Net Present Value (NPV) and Return on Investment (ROI).

Although the concept of a Return on Security Investment (ROSI) is frequently explored in information security literature, there is little consensus on how to calculate it, and few tools to help the ISM do so. 

This article describes a flexible model that merges traditional NPV and ROI calculations into ROSI.  The model employs Monte Carlo simulations to determine the probability that a security investment will produce a positive ROSI. 

A sample Excel spreadsheet demonstrates how this model can be incorporated into a simple-to-use tool. 

Applying NPV to Security Projects

If an organization is considering a traditional business project, it will lay out the costs of the project and the income it provides and compare the NPV of each.  At a very simplified level, the organization determines the ROI of the project with a formula such as:

ROI = NPV of Income Generated by Project – NPV of Project Costs

The same basic approach can be applied to determining the financial feasibility of a security expenditure.  The difference is that security projects never generate income; rather, they prevent the loss of funds that the organization would otherwise devote to operations.

 The value of the project is expressed in the amount of money it “saved” the organization in terms of prevented losses.  Just as with income-generating projects, however, the cost of the security project should be less than the value it provides.  An organization would not spend $50,000 to protect $10,000 in assets. 

ROSI extrapolates the costs that would result from a security breach without a countermeasure in place, the costs of a security breach with the countermeasure in place, and the costs of the countermeasure itself.  The ROSI is then calculated by the formula:

ROSI = NPV of Costs of Unmitigated Breach – NPV of Costs of Mitigated Breach – NPV of Costs of Countermeasure

For example, imagine that a certain security breach would cost a company $100,000 in fines and other cleanup costs at the time of the breach.  A countermeasure is available that would eliminate the risk of fines and reduce the cost of the cleanup to $2,000 at the time of the breach. 

The countermeasure costs $50,000 to purchase and requires $10,000 a year in maintenance.  The discount rate for future cash flows is 5%.  The life of the security countermeasure is five years.

The costs of the project can be analyzed with Excel’s NPV() function.  The spreadsheet Examples.xls (download) shows how this is done. 

In the example shown in the first tab of the spreadsheet, the NPV of the costs of the countermeasure comes to approximately $85,500.  Assume that a breach occurs at the end of the second year (month 24). 

The NPV of an unmitigated breach occurring in month 24 is approximately $90,700, and the NPV of a mitigated breach occurring at the same point is about $1,800.  Using the formula above, the ROSI for this example is calculated as:

ROSI = NPV of Unmitigated Breach - NPV of Mitigated Breach - NPV of Countermeasure
ROSI = $90,700 - $1,800 - $85,500
ROSI = $3,400

So, in this example, implementing the countermeasure saved the organization around $3,400 in today’s money.  However, now imagine that the breach does not occur until the end of the fourth year (month 48). 

The second tab in Examples.xls illustrates this.  In this case, the NPV of the project costs is the same ($85,500), but the NPV of the unmitigated breach is approximately $82,300, and the NPV of the mitigated breach is about $1,600.  Therefore, the ROSI is:

ROSI = $82,300 - $1,600 - $85,500
ROSI = ($4,800)

In this example, the cost of the countermeasure outweighed the cost savings it provided, leading to a negative ROSI.  Of course, if a breach never occurs at all, the entire NPV of the countermeasure becomes a negative ROSI.

Accounting for the Unknown

Because it is impossible to predict when, or if, a breach will occur, the simple ROI method outlined above quickly breaks down.  Nor is this the only uncertainly the ISM must consider.  It is rarely possible to assign a definite cost to a security breach.  

If, for example, a server containing customer PII is compromised, the ISM must consider such hard-to-predict costs as: the staff time to contain the incident and recover from it; contracted forensic investigators; fines; lawsuits; credit monitoring for those affected; lost business; and so on.  None of these costs can be predicted with any certainty.

However, if the ISM can determine or reasonably estimate the ARO of a specific breach, and if the ISM can establish a range of possible costs of that breach, then a model can be constructed to run many thousands of simulations in order to determine the likelihood that a security implementation will result in a positive ROSI. 

A simple Monte Carlo simulation model can be built with Excel’s RAND() and RANDBETWEEN() functions and some fairly simple VBA programming. 

The spreadsheet ROSI_Tool.xls (download) demonstrates how this can be done.  The spreadsheet has five tabs:

  • In the Settings tab, enter the discount rate to be used for the NPV calculations, the ARO of the security breach, and the number of simulations to run.  Nothing should be entered in the other cells. 
  • The ResultChart tab graphically displays the distribution of results of the simulations. Nothing should be entered in this tab.
  • In the third tab, Worksheet-CostOfBreach, project the costs of a breach, starting from the time the breach is discovered and going five years into the future.  For example, you might predict that a certain breach will result in costs due to staff time, the services of forensics consultants to be paid 60 days after the services are rendered, some amount of fines to be assessed and paid a year into the future, and a lawsuit which would probably be resolved after three years of litigation.  These costs are laid out, month-by-month, either as a specific number or as a range of possible values.  Keep in mind that a mitigated breach will still involve some costs, such as the staff time required to investigate the attempted breach and document the outcome.  This tab provides worksheets to lay out the costs of a breach both with and without the countermeasure in place
  • The fourth tab, Worksheet-CostOfCountermeasure, is a similar worksheet.  Here, lay out the costs of purchasing and maintaining the countermeasure.  In addition to the purchase price, you should consider costs such as annual maintenance, staff time to administer, consumption of backup resources and rack space, etc.  In the sample, no variability is built into this worksheet since such costs are typically easier to predict.  However, you may wish to add some random number generation to this worksheet as well.
  • The Simulation tab is the work area for the simulation engine.  As the simulations run, this worksheet calculates the ROSI, which is recorded on the first tab.  Nothing should be entered into the Simulation tab.

Once the worksheets are filled out, the ISM begins the simulations by clicking the button on the Settings tab.  Several thousand simulations will usually compete in less than five minutes.  The most important output of the simulations is the “Percent of Simulations w/ Positive ROSI.” 

This value gives the ISM a good sense of how likely it is that the countermeasure will turn out to be a good investment.  If 75% of the simulations result in a positive ROSI, the ISM can estimate that there is a 75% probability that the security investment will prove to be worthwhile.

The ResultChart graphically displays the results of all the simulations.  This provides the ISM a distribution of possible results and quickly shows the most likely range of outcomes the ISM can expect.

Sample Case

ROSI_Tool.xls (download) demonstrates the following example:

You are managing a system containing PII on thousands of customers.  Your risk analysis and review of threat data leads you to conclude that there is a 33% chance that an external attacker will attempt to access the system in any given year. 

On the Settings tab, you enter 33% for the Annualized Rate of Occurrence (cell B8).  You determine the discount rate for NPV calculations to be 5%, and you enter that in cell B6.

You estimate the costs of a successful penetration with no countermeasures in place:

  • Between $1,000 and $5,000 in staff time to discover and contain the breach at the time it happens;
  • $20,000 to hire a forensics consultant to determine the extent of the breach, with the $20,000 to be paid 60 days after the service is rendered;
  • Between $100,000 and $500,000 in credit monitoring services offered to customers, to be paid 6 months after the breach;
  • Between $500,000 and $2,000,000 in fines from regulators, to be paid one year after the breach;
  • Between $100,000 and $500,000 in legal fees to defend civil suits, to be paid at the conclusion of the trials three years after the breach.

You enter these figures as indicated in the example.  Your risk analysis indicates that the countermeasure will nearly eliminate the possibility of an attacker accessing the PII. 

However, you recognize that an attempted breach still requires a response and may require a detailed analysis to verify and document that the attack was unsuccessful. 

You estimate that an unsuccessful attack would result in $1,000-$3,000 in staff time to respond, $10,000 in outside consulting services, and $5,000 for the legal department to document and report the incident to regulators.  You enter these figures as indicated.

Now, you enter the costs of the countermeasure itself in the tab named Worksheet-CostOfCountermeasure.  The countermeasure will cost $250,000 to purchase and requires a yearly maintenance fee of $50,000. 

In addition, it requires an administrator to spend about 16 hours per month maintaining the system.  The administrator earns $50/hour in salary and benefits, so the administration costs are $800/month.

After entering all the cash flows, return to the Settings tab.  Enter the number of simulations to perform and click the “Run” button (you may have to enable macros).  The spreadsheet will process for a moment.  A message will indicate when the simulations have been completed.

In this example, the countermeasure returns a positive ROSI in about 75-80% of simulations.  The ResultChart indicates that a large number of simulations returned a negative ROSI because the breach did not occur in the five-year system lifecycle. 

However, the total number of simulations returning a positive ROSI is greater than the number of simulations in which the breach did not occur.  Based on this model, you can be fairly confident that the cost of the countermeasure is a good investment.

Conclusion

This model for evaluating security countermeasures attempts to combine the analytical tools of the security professional and the business manager. 

By drawing on security concepts such as Annualized Rate of Occurrence and financial concepts such as Net Present Value, the ISM can select and successfully advocate for security countermeasures. 

The use of Excel makes this a flexible tool that can be modified as needed by the ISM.  Only a basic knowledge of Excel and VBA is required to customize the spreadsheet. 

The model presents a good starting point for the merging of security and business analysis.  However further refinement of the model and the tool will certainly improve the usefulness of both. 

More sophisticated simulation techniques, for example, would lead to greater confidence in the output.  In addition, the graphing of the simulation results could be accomplished more elegantly. 

However, this model does provide a simple and powerful way for the Information Security Manager to evaluate security expenditures and advocate for their implementation in terms that are both meaningful and persuasive to business leaders.

Cross-posted from Kurt Aubuchon's Gather page
Possibly Related Articles:
8407
Budgets
Information Security
Enterprise Security ROI
Post Rating I Like this!
C787d4daae33f0e155e00c614f07b0ee
Robb Reck Great post. This attempt to bring InfoSec decisions more in line with other business decisions is a step in the right direction. Thanks for the impressive leg-work.
1286818671
E68c72e1e8be98215f1fa5155236f5c6
Anthonie Ruighaver I wish the economics of security was that simple. Research in this area of information security has just started and little is known how this approach would actually work in practice were you will have to consider not only what controls to spend your money on, but more importantly how much. I can use a simple product or a more expensive product. Effectiveness of both is uncertain. Should I use the cheap version first and upgrade to the more expensive product only if the increase security is insufficient? Should I use overlapping controls?

Those are some of the issues that security managers have to cope with. There is a real interesting research area of applying Real Options Thinking in Information Security but again the lack of good quantitative data will make it difficult. So maybe we should start with security solutions that provide that data (detective controls for situational awareness, better organizational learning from incident response, etc), but how will we economically justify those solutions?
1286854448
972cda1e62b72640cb7ac702714a115f
Kurt Aubuchon Anthonie:

I agree that security investment decisions must be multifactorial. I would not recommend basing security investment decisions on a single tool, just as a business manager would not rely solely on quantitative anlyses like NPV and ROI when making a business decision. Rather, I think that adding a model like this to the information security manager's tool set, in combination with other decision-making approaches, will help an information security manager address the larger questions you raise.
1286885863
E68c72e1e8be98215f1fa5155236f5c6
Anthonie Ruighaver Well, as I mentioned research in the economics of security is getting more attention now, so expect a few more models for your tool set. I am just not sure this particular model is a good model for use in decision making in an area where uncertainty rules. I'd like to see some more studies of its use in practice.
I am also weary that there is the possibility of misuse, by fudging the figures, if the decision has already been made one way or the other. So, it is important that multiple scenarios are prepared with different sets of figures.
1287028319
959779642e6e758563e80b5d83150a9f
Danny Lieberman Kurt

Excellent article. I've been evangelizing prioritizing and cost-justifying security countermeasures for several years now - what I would add to your model is the notion of asset value to provide a business dimension to the cost engendered by an attack.

Asset value can be estimated fairly quickly by the CFO of a company - not by the IT/IS staffers since they don't have the tools or training.

If you are not familiar - I would suggest taking a look at Practical threat analysis - here http://www.software.co.il/case-studies/254-data-security-threat-assessment.html
and here
http://www.ptatechnologies.com

1287037246
Default-avatar
Dimitrios Stergiou Would it possible to provide an alternate download location for the .xls files, since rapidshare does not seem to host them anymore?

Thanks
1349794688
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.