In the final chapter of the Logging: Opening Pandora's Box (part one: Anxiety) (part two: Elation) (part three: Paralysis) series we'll talk our way through getting to a state where you're actively getting a new level of awareness from out logging capabilities.
Awareness is defined as the ability to know, understand and react to various types of events in near-real-time in order to defend your enterprise. Whether you're defending it from performance failures, functionality failures, or security failures is dependent on the group you work in - but we'll tackle that another time.
Awareness is a difficult thing to attain. It's even more difficult to attain when you have billions (with a b) of log events happening in the enterprise and you don't have a sound strategy for increasing the signal to noise ratio.
The difficulty with getting a decent signal-to-noise ratio is that in an event-centric organizational mindset, everything is critical in a vacuum. Let me explain quickly what I mean by an even-centric organization mindset - this is where every event is investigated in separation from every other event, an application probe from an increase in database CPU, for example.
In a worst-case scenario, every security-related incident such as a ping, a port-scan, or an attempted exploit is categorized and cataloged as a separate event without taking the time to understand the 'bigger picture' of the totality of the possible event. While it's not trivial to link events across disparate systems, platform types, and incident types - it can be done and more importantly is being done today.
Let's analyze each of the three components of a well-balanced awareness strategy...
- Know - Knowing means that your systems, and by extension you, hat the ability to see the events being generated in your organization. This probably means having the ability to collect millions if not billions of log events per day at rapid scale and across disparate systems and non-standard logging formats to accomplish the take of having knowledge. Knowledge in this case may be historical for use when performing empirical historical analysis or used when performing forensics on a breached system. Knowing simply means having the ability to find the information in one place, in an intuitive way.
- Understand - Understanding takes knowledge one step further and starts to connect the dots for your response teams. What does a suspicious-looking application payload (http traffic) have to do with increased CPU cycle time on a database server which is serving the application? Maybe nothing... or maybe it's an indicator that you've been SQL Injected and the attacker is extracting data, or corrupting it... or something else entirely. Understanding ties billions of events, distils them into knowledge that you can use to make sense of your environment and the traffic and usage patterns that show up every day upon careful analysis.
- React - Reacting means making decisions in near-real-time which affect the security (or performance, functional) posture of your enterprise. Knowing you're being probed for an Apache exploit, knowing that exploit is being launched against a patched system, and having the ability to react by not reacting is what a good awareness strategy is all about. Having the ability to react is different than being forced to - as often the appropriate reaction is either to delegate (assign to someone else) or defer (investigate later if other events chain off of this series) or sometimes the appropriate reaction is nothing at all (no risk).
After a roller-coaster ride of emotion stemming from initial discovery of the logging capabilities of most enterprises many security analysts simply give up and resign themselves to having no real-time (or near-real-time) insight into their organization's security posture.
That should be absolutely unacceptable in the modern enterprise where risks can turn on a dime, and today's perceived low-risk attack can quickly escalate into a full-scale incident causing millions or billions of dollars is left unmitigated. This is not FUD, this is simple truth.
The final point to be made here on awareness is having the ability to share that awareness across your internal business units, across the industry vertical your organization belongs to, and with the federal and state governments which are in charge of international incidents of cyber security.
I personally think it's naive to think that kinetic attacks won't be turned into cyber attacks which cause digital and physical damage, or even loss of life.
The threat landscape is entirely too big not to have a microscope that can turn into a fish-eye lens depending on the situation... and remember the key is to be aware in as close to real-time as possible so that you're not left scratching your head after the house has burned down around you.
Cross-posted from Following the White Rabbit