ACL Complexity: How You Are Vulnerable and Do Not Even Know It
Recently we were implementing one of our security products at a new client.
Their prior security program was to use a "top-tier" managed services company and their own network engineer to "co-manage" their security products including their firewalls.
At turn up, our box alerted that it was receiving traffic that was not permitted in the implementation plan. This particular implementation had us behind the firewall instead of our typical implementation outside the firewall. Needless to say, we started asking why this was the case.
Their network engineer opened up their firewalls interface and indicated that the traffic was in fact being torn down and was not passing through. I began to fantasize about how great our company's new business was going to be: creating / crafting network traffic at the quantum level.
Upon further investigation by the managed service company, they too saw that the traffic in question was in fact being torn down and that our box was most likely encountering “false positives”. At this point things got a bit awkward.
The score was two to one on our box being crazy and detecting traffic that was not there. Thankfully, our team was not about to take no for an answer and began supplying traffic information and details to all who would listen.
After three more hours of investigation into the issue, it turns out that conflicting ACLs were to blame.
The interface on their firewall clearly showed that the traffic in question was being “torn down”, but from our logs it was passing through. Only after hours of effort and hunting did the issue finally get discovered and subsequently resolved.
To us this discovery was terrifying. If the only way to tell if the ACLs are properly configured is to use another detection mechanism that is capable of identifying improper traffic and nobody had anything like that on their networks, then how many networks are completely vulnerable and do not know it?
Like most companies, our new client had their ACLs built organically over time. Let's face it, not many fancy a full day or two of ACL design perfection. Not to mention ever finding the time to make such effort.
Due to the organic nature of the ACL growth and the changing requirements for different systems / rollouts / etc., many network admins are never afforded the opportunity to sanity check their rulesets. Even worse, if they go to investigate the issue, their firewall will give them false assurance.
It begs the question, how big is this problem?
In our experience, this real life example is just the tip of the iceberg. We frequently encounter unexplained phenomena at the networking / security level and all to often it is ultimately ignored. All too often, when a misconfiguration or other mishap is discovered the first rule of action is to attack the messenger.
The net result is that there are many companies out there just waiting to be breached. Most of them could be prevented.
It is our hope that bringing this real world example to light will cause network admins everywhere to take a good hard look at their ACLs and make sure they do not live under the same illusion.
