Ending the Security Business of Guessing

Tuesday, July 13, 2010

Pete Herzog


What if a Risk Assessment is no longer the best way to evaluate an organization or prepare it for future threats? What if there was a better way? Would you change?

In the research for factual security metrics, factual trust metrics, and reliable, repeatable ways for verifying security, including concretely defining security, we found that the practice of guessing forecasting risk was not only non-factual but also backwards. Risk stuck us into a never-ending game of cat and mouse with the threats.

It made me wonder how that happened in the first place. Is it really because security is a process or is it because humans are reactionary creatures who can't see beyond the latest threat? Or, and this is me just reaching here, could it be that reactionary and process-based security is good for the business of selling security?

Beginning with version 3, the OSSTMM is no longer just about security testing. The break-throughs we've had in security had us re-visit how we work with security. This includes risk assessments.

Now what if you never had to do another risk assessment again? What if it meant no more forecasting potential threats, scanning for vulnerabilities, patching systems, and weighing it all for impact? What if you never had to guess analyze whether or not the latest vulnerability or exploit buzz will mean another long day of putting out fires for you?

While that scenario goes against nearly everything that's taught in security these days, there is evidence that this is not only possible but easier and cheaper to implement. Wow, it didn't sound like a late-night infomercial until I just wrote it....

The new OSSTMM teaches how to test and analyze anything so as to know where the interactions are and how those interactions are being controlled. It even shows you how to measure how much reason you have to trust the agents of those interactions so as to determine where you want to put most of the money into controls.

It even shows you which specific controls you need to put there and how to evaluate products to know the operational controls they will provide and where before you even buy the product, allowing you to avoid wasting money on controls which are already redundant or, as we have found to be the real problem, controls which don't provide the operational safety how and where they say they will on the box.

Okay, if it sounds far-fetched, let me put it another way. You can know to a decent certainty what your assets are. You can know to that same certainty how your operations run. If you can know which controls govern those operations, you can also know which are missing.

Since there are only 10 possible operational controls, you can close holes before they are known holes and render whole classes of threats moot. You also don't need to worry about 0-days in your systems because you assumed already they were there and put solid, operational controls on interactions with those applications and separate those you can't protect. We call this Defense in Width.

But it's not over. See, organizations change, the world turns, and other things happen. Now you may think, well security IS a process. But that's not true. Your security is done. The problem is trust. Trust is a process. As things change, so will your reasons to trust.

So you will be changing your controls based on changes in operations and changes in trust over those interactions. You need to address trust issues before they become problems. You need to integrate operational trust metrics into hiring new employees as well as the regular evaluations of employees. You need to evaluate old and new partners continuously.

This the process you will need to adjust to. And it will be easier and cheaper to integrate this with much less error then guessing evaluating present and future risks can ever give you if only because there is no guessing involved. None at all.

So would you change if there was something better? You may want to consider it especially if your customers might be looking for it and if they can't get it from you they'll need to go elsewhere. You'll find the answers in OSSTMM 3, maybe even already practiced by security professionals near you.

Possibly Related Articles:
Information Security
Risk Assessments OSSTMM
Post Rating I Like this!
Mister Reiner Just some thoughts off the top of my head...

This certainly sounds interesting, but I think you need to explain it in layman's terms and provide some examples of what you're talking about to bridge the knowledge gap between the security gurus who know OSSTMM and those not familiar with it. People understand the terms "threats", "vulnerabilities" and "patching", but I think you start to lose people when you start using terms like "interactions", "controls" and "trust", because those things mean different things to different people. Otherwise, it just comes across as some type of mystic black box process that sounds too good to be true.
Pete Herzog Thanks for the input Mister Reiner! I had a much longer blog text prepared but the problem is always the same, communicating new ideas and philosophies will always strain a vocabulary. Especially since the security literature is full of different definitions for various concepts and various security professionals already assume different definitions for others (like Defense in Depth). I hope that the OSSTMM 3 detail will help people. This is just to get them interested in knowing what's out there that can really help.
Marc Doudiet Dear Pete,

OSSTMM seems a very interesting concept, but sorry to say that it is only a concept.
I follow the news regarding version 3 for at least 3 years, 3 years in IT is a lot too long without anything to use.
I don't have a piece of paper even to try to correlate with my actual way of assessment, or I have to pay the fees (that are strange for an open source standard) that could point me to a draft only. Even the courses are really expensive for a standard that is not mature enough to be released in the public.

Sorry to say that, because I really think that the actual information risk assessment methods are not mature and that a new approach could be very interesting but I don't see anything usable in the OSSTMM.

Best regards.
Pete Herzog Marc, thanks for the honest feedback. Our Debian servers say 2006 and are running fine without problem. That's 4 years. The 2 Windows XP PCs we have in our office say 2002. They run everything that Windows 7 runs and are probably even more secure and stable. That's 8 years. My ISO 27001 security standard says 2005. That's 5 years. While I think technology might move fast in some areas (hardware for example) in others, much of it is the same but with new marketing. Really, most security concepts still in use date back to the 1970s. So while I find technology to move fast, research still takes time to be done and for concepts to get tested. We've been doing both. That takes time. But it's also not hardware dependent so we're fairly safe there. Currently, the supporters and collaborators on ISECOM projects have access to the OSSTMM. It will be open and free to the public when it's done. That still sounds like Open Source to me. You can get involved and get access now too if you want. We're an open community and don't discriminate against any valuable contributions. Additionally, we've been sharing it with various government entities and ISO to use the draft not because the concepts are immature but because the document was unedited. For example NIST had a rough copy years ago for writing NIST-SP800-42. Whole parts are completed, like the ones the classes are based on, but there is a lag between making the information and having the best way to convey it. I am often reminded by a posting to a list that OSSTMM 2 read like "stereo instructions" which is something we've been working really hard on improving. So because a document isn't publishable doesn't mean the ideas and concepts are bad. It sounds to me like you're frustrated that the OSSTMM isn't public yet. I'm frustrated about that too but we're working the best we can on getting it there- believe me, we have a lot of pressure! Meanwhile, anything you can take over for us (wanna write a Hacker Highschool lesson?) could help us move forward and will get you access to OSSTMM 3. I don't see how that's unreasonable.
Koen Bossaert Marc, have a look at SABSA, a methodology for entreprise security architecture. It exists for more than 10 years now and only since last year it's getting traction and more involvement. OSSTMM is going a lot faster compared.
Nigel Mellish Koen - are you serious? SABSA? Why would you wish that beast of a document on anyone who isn't paid by the hour?

Pete - You can rename risk to trust or whatever fuzzy name you want, but it's still risk. Making a probabilistic determination of trust is still a risk analysis. Frankly, OSSTMM would be a lot more credible if it weren't trying so hard to market itself as unique vs. solve the problems we already know exist. Until then, playing semantic games only creates more equivocality.
Pete Herzog Hi Nigel. I guess I did a poor job of explaining here that we want to get rid of probabilistic anything and rely only the facts at hand to measure our current capabilities for the future and not predict the future. So this isn't me marketing anything by changing the name of risk to trust. It's me explaining how we were able to derive factual trust metrics which will allow us to improve what risk assessments are currently used for. One key to that is why we call it operational trust or "reasons to trust" instead of just trust because it isn't fuzzy at all. But you bring up another point when you mention you wouldn't wish SABSA on anyone. We want the OSSTMM to be readable and educational for anyone and make it something people will want to read not be "assigned" to it like some prison duty. That's one of the reasons so much work has gone into it to prepare it for production.
Nigel Mellish Wait, "rely only on the facts at hand"? How do you know that you're looking at a representative sample?

Unknown unknown problems are well known (and documented) to contribute to security incidents. I'm concerned that what you're proposing is a head in the sand strategy concerning the threat and asset landscapes. Based on assumptions that you have a complete control landscape and a 100% effectiveness for control operations.
Koen Bossaert Nigel: I don't see how your remark relates to my comment. My only point was to say that it can indeed take a bit of time to develop and mature a methodology. SABSA can be realy heavy indeed, I agree on that.
Pete Herzog Nigel, don't worry. We're not putting our heads in the sand for anything. We are indeed looking at the representative sample because we can only measure what we have. If we cannot verify it then we cannot measure it factually. I do suggest you read more details on it before you begin to lose sleep over it. Although the OSSTMM 3 has not been released yet publicly, you can find information from presentations here: http://www.isecom.org/news.shtml

I do want to point out though that we were very aware of this type of reaction and are trying very hard to make the OSSTMM concepts as clear as possible to help people understand the new information.
Nigel Mellish ohhh, so you're just acknowledging the fact that there's known information with uncertainty, and ignoring it.

It's pretty much the frequentist/objectivist trap IJ Good alluded to. Gotcha. Good luck with that.
Nigel Mellish Also,

"rely only the facts at hand to measure our current capabilities for the future and not predict the future."

Pete Herzog Nigel, I have to admit I no longer follow you. I think you're twisting around things which I hadn't answered with things I had. What I know of "known info with uncertainty" is that of operations which have not happened yet. The OSSTMM covers operational security which means we verify operationally how things work now under a thorough battery of tests. I'm not sure if you knew that. So we can't predict the future but we can say what the capabilities are now based on the results of operational tests and determining which operational controls are in place and what weaknesses they have. This is preparing for the future and not predicting it. How is that sticking our head in the sand? In traditional risk assessments you guess what the future threats will be and essentially stick your head in the sand based on what you conclude can't happen. We don't do that at all. So I think you're a bit confused here. Well, actually, more than a bit....
Rob Lewis Hi Pete,

Nice post. You are hitting on the cusp of an unarticulated problem in Infosec; there is a huge disconnect between the business rules (operational privileges) and IT security policies. John Pescatore of Gartner once wrote in a discussion on his blog that in his opinion this gap was major and that it would never be bridged.

Part of the problem is that the business rules use trust relationships (and language) to denote relative risk within and between enterprise user groups and partners, while IT security policies and risk models are still object-centric.

It is intuitive to sense that some sort of provisioning should exist so that new hires don't get the keys to the kingdom, and to relate to a very recent poll and article in the media, that IT staff should not be able to access sensitive data just because they have certain passwords.

You are on the right track. Let's face it, the status quo sure isn't working. All that is needed is a means to place controls where business rules require them, and confirm that those rules were enforced.

chit heer Pete: some times I have to remind my-self " People just don't get it" . The days of manual work are almost extinct ..globally.
We have a whole new vocabulary.....Job related jargon, innovations of "management guru's".
In the end people have put their faith in processes and machines and hence the trust factor has diluted.
Security should be about good plain old common sense, how can you predict where the enemy is going to attack or when... (unless you've set a date for a match).
Trust on the other hand always need to be built upon. Hence it is easy to understand why it is a process.
Unfortuntley fear is what drives most people not dreams...(Some might think that is a wild swing but I hold it as a valid point)....
....Having become SO educated we have forgotten the simple realities...I remember in the old day of the 486 era fantsizing about the security issues we face today..but who listened....
To me OSSTMM 3 seems to be taking the direction that people who values goals and have a vision for the future will adopt... I will without doubt.
Pete Herzog Thanks for the support, Chit! One of the things I learned about trust is that people are bad at seeing past the first level of a trust- what's behind it. It's the reason why commercials with actors dressed at doctors or housewives or anyone we feel comfortable or familiar with tell us work on the masses. They know it's an actor but they feel a trust connection. It's a disassociation between the brain and the heart, so to speak, and the heart wins by making an itch we automatically want to scratch. So it's also why trusted computing works because little thought goes into the humans designing the components which make the machine. There are many hands on every machine BEFORE it's sealed and called "trusted". People accept that because they don't look past the first level of trust which is with the organization selling the trusted systems. To use operational trust we need make sure it comes from a place of reason and we need to gauge all 10 properties of trust, of which 1 is Components, the previous interactions with the pieces that makes up the whole. It helps us see through the fabrications, illusions, and marketing hype for what it is. That's probably why Certified Trust Analyst seems to be our hottest selling class right now. Strangely, very few security people sign up for it when they could certainly need it as well.
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.