Often, the decision of whether or not to implement a security countermeasure is ultimately based on a vague appeal to "best practices", or on a gut feeling that the cost of the countermeasure is justified by the risk of an exposure.
In this article, I propose a model, based on Net Present Value, Return on Investment, and Monte Carlo simulations, that provides a quantitative framework for these decisions. A sample analytical tool is also provided.
Justifying the costs of security expenditures is a persistent challenge for the Information Security Manager.
The often significant implementation and maintenance costs of a countermeasure, coupled with the difficulty of quantifying the costs of a breach, make traditional cost-benefit analyses problematic at best, and simple guesswork at worst.
While information security literature addresses such concepts as Single Loss Expectancy (SLE), Annualized Loss Expectancy (ALE), and Annualized Rate of Occurrence (ARO), there are few good models available to translate these concepts into the kind of business analyses that are persuasive to operational managers.
In addition, the models that are available often fail to adequately take into account the probability and timing of a security breach. An extravagant SLE will fail to persuade senior executives who perceive a vanishingly low probability that such an event will occur.
To be successful, the Information Security Manager (ISM) must be able to make a case for security investments in terms that are persuasive to business managers, not necessarily to other information security professionals.
Senior management will be unmoved by appeals to IT best practices; they will become inured to frightening tales of companies who paid the price for lax security practices; and they will lose confidence in a security manager who supports proposals with little more than instinct.
The ISM must be able to draw on traditional analytical tools such as Net Present Value (NPV) and Return on Investment (ROI).
Although the concept of a Return on Security Investment (ROSI) is frequently explored in information security literature, there is little consensus on how to calculate it, and few tools to help the ISM do so.
This article describes a flexible model that merges traditional NPV and ROI calculations into ROSI. The model employs Monte Carlo simulations to determine the probability that a security investment will produce a positive ROSI.
A sample Excel spreadsheet demonstrates how this model can be incorporated into a simple-to-use tool.
Applying NPV to Security Projects
If an organization is considering a traditional business project, it will lay out the costs of the project and the income it provides and compare the NPV of each. At a very simplified level, the organization determines the ROI of the project with a formula such as:
ROI = NPV of Income Generated by Project – NPV of Project Costs
The same basic approach can be applied to determining the financial feasibility of a security expenditure. The difference is that security projects never generate income; rather, they prevent the loss of funds that the organization would otherwise devote to operations.
The value of the project is expressed in the amount of money it “saved” the organization in terms of prevented losses. Just as with income-generating projects, however, the cost of the security project should be less than the value it provides. An organization would not spend $50,000 to protect $10,000 in assets.
ROSI extrapolates the costs that would result from a security breach without a countermeasure in place, the costs of a security breach with the countermeasure in place, and the costs of the countermeasure itself. The ROSI is then calculated by the formula:
ROSI = NPV of Costs of Unmitigated Breach – NPV of Costs of Mitigated Breach – NPV of Costs of Countermeasure
For example, imagine that a certain security breach would cost a company $100,000 in fines and other cleanup costs at the time of the breach. A countermeasure is available that would eliminate the risk of fines and reduce the cost of the cleanup to $2,000 at the time of the breach.
The countermeasure costs $50,000 to purchase and requires $10,000 a year in maintenance. The discount rate for future cash flows is 5%. The life of the security countermeasure is five years.
The costs of the project can be analyzed with Excel’s NPV() function. The spreadsheet Examples.xls (download) shows how this is done.
In the example shown in the first tab of the spreadsheet, the NPV of the costs of the countermeasure comes to approximately $85,500. Assume that a breach occurs at the end of the second year (month 24).
The NPV of an unmitigated breach occurring in month 24 is approximately $90,700, and the NPV of a mitigated breach occurring at the same point is about $1,800. Using the formula above, the ROSI for this example is calculated as:
ROSI = NPV of Unmitigated Breach - NPV of Mitigated Breach - NPV of Countermeasure
ROSI = $90,700 - $1,800 - $85,500
ROSI = $3,400
So, in this example, implementing the countermeasure saved the organization around $3,400 in today’s money. However, now imagine that the breach does not occur until the end of the fourth year (month 48).
The second tab in Examples.xls illustrates this. In this case, the NPV of the project costs is the same ($85,500), but the NPV of the unmitigated breach is approximately $82,300, and the NPV of the mitigated breach is about $1,600. Therefore, the ROSI is:
ROSI = $82,300 - $1,600 - $85,500
ROSI = ($4,800)
In this example, the cost of the countermeasure outweighed the cost savings it provided, leading to a negative ROSI. Of course, if a breach never occurs at all, the entire NPV of the countermeasure becomes a negative ROSI.
Accounting for the Unknown
Because it is impossible to predict when, or if, a breach will occur, the simple ROI method outlined above quickly breaks down. Nor is this the only uncertainly the ISM must consider. It is rarely possible to assign a definite cost to a security breach.
If, for example, a server containing customer PII is compromised, the ISM must consider such hard-to-predict costs as: the staff time to contain the incident and recover from it; contracted forensic investigators; fines; lawsuits; credit monitoring for those affected; lost business; and so on. None of these costs can be predicted with any certainty.
However, if the ISM can determine or reasonably estimate the ARO of a specific breach, and if the ISM can establish a range of possible costs of that breach, then a model can be constructed to run many thousands of simulations in order to determine the likelihood that a security implementation will result in a positive ROSI.
A simple Monte Carlo simulation model can be built with Excel’s RAND() and RANDBETWEEN() functions and some fairly simple VBA programming.
The spreadsheet ROSI_Tool.xls (download) demonstrates how this can be done. The spreadsheet has five tabs:
- In the Settings tab, enter the discount rate to be used for the NPV calculations, the ARO of the security breach, and the number of simulations to run. Nothing should be entered in the other cells.
- The ResultChart tab graphically displays the distribution of results of the simulations. Nothing should be entered in this tab.
- In the third tab, Worksheet-CostOfBreach, project the costs of a breach, starting from the time the breach is discovered and going five years into the future. For example, you might predict that a certain breach will result in costs due to staff time, the services of forensics consultants to be paid 60 days after the services are rendered, some amount of fines to be assessed and paid a year into the future, and a lawsuit which would probably be resolved after three years of litigation. These costs are laid out, month-by-month, either as a specific number or as a range of possible values. Keep in mind that a mitigated breach will still involve some costs, such as the staff time required to investigate the attempted breach and document the outcome. This tab provides worksheets to lay out the costs of a breach both with and without the countermeasure in place
- The fourth tab, Worksheet-CostOfCountermeasure, is a similar worksheet. Here, lay out the costs of purchasing and maintaining the countermeasure. In addition to the purchase price, you should consider costs such as annual maintenance, staff time to administer, consumption of backup resources and rack space, etc. In the sample, no variability is built into this worksheet since such costs are typically easier to predict. However, you may wish to add some random number generation to this worksheet as well.
- The Simulation tab is the work area for the simulation engine. As the simulations run, this worksheet calculates the ROSI, which is recorded on the first tab. Nothing should be entered into the Simulation tab.
Once the worksheets are filled out, the ISM begins the simulations by clicking the button on the Settings tab. Several thousand simulations will usually compete in less than five minutes. The most important output of the simulations is the “Percent of Simulations w/ Positive ROSI.”
This value gives the ISM a good sense of how likely it is that the countermeasure will turn out to be a good investment. If 75% of the simulations result in a positive ROSI, the ISM can estimate that there is a 75% probability that the security investment will prove to be worthwhile.
The ResultChart graphically displays the results of all the simulations. This provides the ISM a distribution of possible results and quickly shows the most likely range of outcomes the ISM can expect.
ROSI_Tool.xls (download) demonstrates the following example:
You are managing a system containing PII on thousands of customers. Your risk analysis and review of threat data leads you to conclude that there is a 33% chance that an external attacker will attempt to access the system in any given year.
On the Settings tab, you enter 33% for the Annualized Rate of Occurrence (cell B8). You determine the discount rate for NPV calculations to be 5%, and you enter that in cell B6.
You estimate the costs of a successful penetration with no countermeasures in place:
- Between $1,000 and $5,000 in staff time to discover and contain the breach at the time it happens;
- $20,000 to hire a forensics consultant to determine the extent of the breach, with the $20,000 to be paid 60 days after the service is rendered;
- Between $100,000 and $500,000 in credit monitoring services offered to customers, to be paid 6 months after the breach;
- Between $500,000 and $2,000,000 in fines from regulators, to be paid one year after the breach;
- Between $100,000 and $500,000 in legal fees to defend civil suits, to be paid at the conclusion of the trials three years after the breach.
You enter these figures as indicated in the example. Your risk analysis indicates that the countermeasure will nearly eliminate the possibility of an attacker accessing the PII.
However, you recognize that an attempted breach still requires a response and may require a detailed analysis to verify and document that the attack was unsuccessful.
You estimate that an unsuccessful attack would result in $1,000-$3,000 in staff time to respond, $10,000 in outside consulting services, and $5,000 for the legal department to document and report the incident to regulators. You enter these figures as indicated.
Now, you enter the costs of the countermeasure itself in the tab named Worksheet-CostOfCountermeasure. The countermeasure will cost $250,000 to purchase and requires a yearly maintenance fee of $50,000.
In addition, it requires an administrator to spend about 16 hours per month maintaining the system. The administrator earns $50/hour in salary and benefits, so the administration costs are $800/month.
After entering all the cash flows, return to the Settings tab. Enter the number of simulations to perform and click the “Run” button (you may have to enable macros). The spreadsheet will process for a moment. A message will indicate when the simulations have been completed.
In this example, the countermeasure returns a positive ROSI in about 75-80% of simulations. The ResultChart indicates that a large number of simulations returned a negative ROSI because the breach did not occur in the five-year system lifecycle.
However, the total number of simulations returning a positive ROSI is greater than the number of simulations in which the breach did not occur. Based on this model, you can be fairly confident that the cost of the countermeasure is a good investment.
This model for evaluating security countermeasures attempts to combine the analytical tools of the security professional and the business manager.
By drawing on security concepts such as Annualized Rate of Occurrence and financial concepts such as Net Present Value, the ISM can select and successfully advocate for security countermeasures.
The use of Excel makes this a flexible tool that can be modified as needed by the ISM. Only a basic knowledge of Excel and VBA is required to customize the spreadsheet.
The model presents a good starting point for the merging of security and business analysis. However further refinement of the model and the tool will certainly improve the usefulness of both.
More sophisticated simulation techniques, for example, would lead to greater confidence in the output. In addition, the graphing of the simulation results could be accomplished more elegantly.
However, this model does provide a simple and powerful way for the Information Security Manager to evaluate security expenditures and advocate for their implementation in terms that are both meaningful and persuasive to business leaders.Cross-posted from Kurt Aubuchon's Gather page