Getting Past Security's Fuzzy Math ROI

Monday, December 05, 2011

Rafal Los


"An unsophisticated forecaster uses statistics as a drunken man uses lamp-posts - for support rather than for illumination." --Andrew Lang [1]

That quote by the mid 1800's Scottish poet perfectly illustrates how I feel about providing a security ROI

It seems that we're using statistics, metrics, surveys and 'studies' to demonstrate what we can't otherwise adequately explain. That would be all well and good, if the math wasn't all fuzzy

I'm not saying my colleagues across the industry are making up these figures they're using to justify products, services, or what-not... because numbers can't fib, only the people that manipulate them can be accused of that trespass. 

No, we aren't fabricating truths, but I strongly feel as if we're making stretches where they really, really don't work.  We see ourselves as being forced to justify our existence, justify our future expenditures and projects - so we create a fuzzy science around Information Security ROI.

It's hurting the credibility of the broader information security and risk management function (as a business function) and it's making me crazy.

If you've seen any of my Security BSides talks, the last few have been titled "The Business Doesn't Care... and it's your fault" and no one will accuse me of being kind to the information security practitioners and management who can't figure out how to communicate with the organization they're supporting. 

Sure, it's not easy.  Sure, communication isn't something you learn from being an InfoSecurity Troll but it's also not rocket science.  Communication is key in almost every aspect of life from relationships to business... so what gives?

Look, if I want an organization to believe that they need to buy more antivirus, I simply have to show them some scary figures, and find some numbers somewhere which can conclusively prove, via an ROI analysis, that having (more) anti-virus will save the company money in the long run. 

OR I can simply work to demonstrate that productivity increases shareholder value, and linking that productivity to various proven KPIs which may not have an ROI, but they'll certainly illustrate a point.  What's more, ROI (return on investment) isn't always possible when we are essentially selling our organizations on the equivalent of life insurance.

Let me give you a solid example.  Since I'm familiar with the software security world, let's look at the ROI of implementing a software security program.  A return on investment has to be calculated against something else... and the fallback of everyone I've talked to on this topic from the business perspective is "do nothing". 

If I do nothing, we consider that a baseline, so how does spending $1,000,000.00 become a return on investment.  Do you suddenly get faster development cycles?  No, in fact you'll get the opposite for quite a while.  What can we calculate as the return on that $1M investment?  Safer software... right? 

Sure, but just how do you calculate the impact of 'safer software'.  This is where the fuzzy math comes in, and we start making up formulas that demonstrate that in the event of a breach, we will have to forfeit $x profit, lose $x in shareholder value and so on... but here's the harsh reality. 

Even if we spend that $1M on the security program to have more safe software - we will likely still get breached and have losses... so what happens to that ROI?  Can you say with any certainty that less vulnerabilities leads to less breaches?  I'm not willing to take that point for granted.  So now what?

You can probably see the complexity in calculating a return on investment (ROI) for something like software security, but the same rules apply to everything because in security there are no absolutes.  Spending more money doesn't necessarily make you safer, or less likely to have a serious impact incident.  Just like spending less doesn't necessarily mean you'll get hit more, or it's any more likely.

So maybe if we spend the right amount of money on a better product/service we will get a real return on our investment?  No... because the word better is in that sentence.  How do you define better?  More importantly, better in what context?

As you can see, the idea of a security ROI gets lost quite quickly in details, and while I can make a compelling ROI on the surface - if you start digging into my methodology you'll quickly realize I'm either making assumptions you're not comfortable with, or manipulating facts to my own ends... and I haven't seen an ROI that doesn't do one of those two.

So sanity, please... now what?

In my opinion we need to go back to having business value conversations, and communicating more effectively than we have been in the past.  Information Security is a business enabling function, and if your organization doesn't understand that fact - it's your fault.  You need to learn to communicate effectively, demonstrate real risk vs. value, and get away from hocus-pocus.

How does that software security program add to shareholder value?  I can list off a few examples, but they likely won't apply to your business value, and what your shareholders and organization care about. 

In a hospital, software security increases reliability and availability - which leads to a higher patient survivability rate.  I bet if you work at a hospital you care about patient survivability... maybe not in the Information Security group, but certainly at the leadership, management, and executive levels.  Now, go and apply that to your organizational goals.

Before you start another project ask yourself... 'How does this increase shareholder (or stakeholder, if you don't have any shareholders) value?  If you don't know the answer to that question... or you have to draw some uncomfortable connections - STOP... you're probably doing it wrong.

This will be the topic of a future podcast, so I'm giving you adequate time to think about it, get yourself fired up, and come up with some arguments, examples, and ideas to discuss with.  Let's take the magic out of Information Security so we can deliver some real business value.


[1] Andrew Lang., Explore Inc, 2011., accessed November 12, 2011.

Cross-posted from Following the White Rabbit

Possibly Related Articles:
Enterprise Security
Information Security
Enterprise Security Budgets ROI Vendor Management metrics Information Security ROSI
Post Rating I Like this!
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.