How Do You Know When You’re Done with an Incident? A Lesson from the Nortel Networks Breach
According to the Wall Street Journal, the high tech company Nortel Networks was breached by Chinese hackers, who were able to retain that access for upwards of ten years.
The initial breach vector is said to have been seven compromised passwords, and the only remediation taken was to change those passwords.
Further recommendations to increase security controls were vetoed by senior management, according to the report. The investigation was closed after six months, due to a lack of leads.
Much is being published about how inappropriate the response to this incident was, but it does demonstrate an important point for companies to consider—how do you know when you’ve done enough? How do you tell when an incident is over, and you should go back to business as usual?
Ideally, this information should be in the organization’s incident response plan. However, many plans are more focused on the response and containment phases and don’t have well-defined criteria for ending an incident.
If, as the article suggests, there were indications that machines were sending traffic to addresses in China (or anywhere else that was deemed non-business related and likely malicious), at the very least, that should have pointed to a continued presence of malicious actors.
Clearing the malware off of the offending systems was apparently not successful enough if the data loss continued—or perhaps it was that the systems identified were simply jumping off points to bigger and better access. Either way, containment was not achieved.
At the very least, if there has been an indication of data exfiltration, and the organization had the ability to alert on such traffic (which is implied), detective alerts should have been put in place so that if the traffic started up again it would let them know.
One problem is that (and this is particularly true when the system(s) involved are production, mission critical, or otherwise important) in the midst of an incident, there can be pressure from the business to ”just get the systems back up and running as soon as possible” to avoid the downtime consequences.
If this pressure is too high on those performing the incident response, it can lead to the case where the full extent of the compromise is not discovered.
However, even in the case of pressure to restore, other mitigating detective controls should be put in place to help detect a renewal of the indicators of compromise that alerted the team to the problem originally.
The WSJ article indicates that the intruders had the run of the place at Nortel, and downloaded sensitive intellectual property, emails, and other data. This implies the traffic would have continued (although it could have morphed in subsequent transmissions) and something should have been detectable.
So let this be a lesson to those of you who are responsible for incident response planning—make sure there are defined criteria that helps your team make the call (despite outside pressure) to end an incident only when all indicators of compromise have ceased, and to ensure that detective measures are in place in case the activity starts up again.
Have these criteria approved by senior management as part of the plan, so that they can help deflect the pressure from the team doing the investigation until the organization has a reasonable expectation that they have completed their due diligence.