SMTP Dialects: How to Detect Bots Looking at SMTP Conversations

Wednesday, October 03, 2012

Gianluca Stringhini


It is somewhat surprising that, in 2012, we are still struggling fighting spam. In fact, any victory we score against botnets is just temporary, and the spam levels raise again after some time. As an example, the amount of spam received worldwide dropped dramatically when Microsoft shut down the Rustock botnet, but has been rising again since then.

For these reasons, we need new techniques to detect and block spam. Current techniques mostly fall in two categories: content analysis and origin analysis. Content analysis techniques look at what is being sent, and typically analyze the content of an email to see if it is indicative of spam (for example, if it contains words that are frequently linked to spam content). Origin analysis techniques, on the other hand, look at who is sending an email, and flag the email as spam if the sender (for example the IP address the email is coming from) is known to be malicious. Both content analysis and origin analysis techniques fall short and have problems in practice. For instance, content analysis is usually very resource intensive, and cannot be run on every email sent to large, busy mail servers. Also, it can be evaded by carefully crafting the spam email. On the other hand, origin analysis techniques often have coverage problems, and fail to detect as malicious many sources that are actually sending out spam.


In our paper B@BEL: Leveraging Email Delivery for Spam Mitigation, that got presented at the USENIX Security Symposium last August, we propose to look at how emails are sent instead. The idea behind our approach is simple: the SMTP protocol, which is used to send emails on the Internet, follows Postel's Law, which states: "Be liberal in what you accept, but conservative in what you send". As a consequence of this, email software developers can come up with their own interpretation of the SMTP protocol, and still be able to successfully send emails. We call these variations of the protocol SMTP dialects. In the paper we show how it is possible to figure out which software (legitimate of malicious) sent a certain email just by looking at the SMTP messages exchanged between the client and the server. We also show how it is possible to enumerate the dialects spoken by spamming bots, and leverage them for spam mitigation.


Although not perfect, this technique allows, if used in conjunction with existing ones, to catch more spam, and it is a useful advancements in the war against spamming botnets.


Gianluca Stringhini is a PhD candidate working as research assistant at UC Santa Barbara. His research interests are Network Security, Botnets, and Spam Mitigation. You can follow him on Twitter at @gianlucaSB

Possibly Related Articles:
Firewalls IDS/IDP Network Access Control Network->General SCADA SPAM Viruses & Malware Phishing
Information Security
SPAM Methodologies Monitoring SMTP bots
Post Rating I Like this!
Marc Quibell I would say the reason SPAM is still around is because it's just easier to toss it out, rather than even worry about it. Same reason junk mail still arrives in our mail boxes.
Gianluca Stringhini Marc you are right, but spam is also a waste of resources for mail servers... being able to quickly assess whether an email is spam is important for the email delivery infrastructure.
Marc Quibell That's pretty much what SPAM filters do today: content and origin analysis. And, on the mail servers, still wasting resources. Now if you could move filters closer to the source...there's an idea!
Gianluca Stringhini The idea is to avoid content analysis whenever possible. And since origin analysis doesn't always work, one can use our technique to aid origin analysis. And our technique is lightweight as well
Chih-Cherng Chin We all know that botnets send more than half of global spam. So to reduce spam, the objective is not to catch more spam, but to decrease the number of infected computers. And merely detecting spambots is not enough, we should notify the victims so that they can clean up their computers. Without notification, the data is not being put to good use.
This is certainly not easy, but it's doable. I know it because I have been detecting spambots for 3 years, with "follow the spam" strategy (via fake open relay and greylisting), and notifying the victims through ISPs and CERTs. I recrod my detection and notificaton data at .
Gianluca Stringhini There are several approaches to fight spam and infections, the one of sending notifications is one, but you need a lot of interaction between the different ISPs, that is hard to get sometimes.
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.