Bleeding Edge Technology Finds Spammers Where They Live

The short history of NSBL

In 2001, April Lorenzen was the co-owner of a web/email hosting ISP company who was growing frustrated with the available spam-fighting tools. "As a hosting ISP responsible for several hundred business domains in the late 1990s, we always employed moderate anti-spam techniques," April says. "However, our frustration level mounted as customers complained about the amount of offensive and annoying spam email that still leaked through daily."

So she started experimenting with radical approaches—such as parsing the Spamcop.net web page every 5 minutes, and automating a 24-hour block on the addresses found there. This method was neither 100% effective nor free from false positives. But she did notice, as part of this experiment, that spammers were skipping over a large and unpredictable variety of Class C netblocks, often only staying on a particular IP for a few hours.

The light bulb goes on

In June of 2001, April had an epiphany—she realized she was wasting her time chasing after lists of 'bad' IPs that would never stop growing and changing. Spammers have nearly 2 billion possible addresses at their disposal, so they won't run out of fresh unlisted IP addresses any time soon. Instead, it seemed far more efficient to create a worldwide index of domains and their associated, authorized outbound email servers, stored in a shared repository database that also contained other information useful to those fighting spam.

So she formed a new corporation to develop a reliable, scalable infrastructure for the concept of an email server identity database. She called it the Outbound Server Authentication Index (Outbound Index, for short). The people and resources needed to create and further refine the prototype came mostly from the Open Source community. Her team rapidly created a query-response server that interfaced to the prototype Outbound Index database and worked with Postfix and Sendmail. Now properly-configured mail servers could query this repository, and automatically reject mail with either a forged return address or one from servers operating on IPs forbidden by their own ISPs.

After that, they quickly realized they could add other reputation checks to the same infrastructure. Over the following years, they started to expand the checklists, focusing closely on spammer-like domain registration and reputation.

Some of the additional checks included:

• Has a trusted SSL certificate on its website?
• Has registration and usage patterns that match characteristics of a throw-away domain?
• Shares name server hosts, whois data, registrar, etc. with throw-away domains?

Using this new reputation data, they found the data to be very effective. With it, they could help ISPs make very accurate decisions as to whether messages were coming from a spammer or other cyber-miscreant. The beauty of the approach is that it can detect bad domains days or even weeks before these domains are actually used as part of a global spam campaign. This headstart is a huge advantage in boosting the effectiveness of any security infrastructure.

In 2009, Messaging Architects started to work with April to further refine the concept and the reputation scores so they would also work well with corporate mail server patterns. We then added support for the feed directly as part of our Netmail engine and coined the new solution as NSBL, short for Name Server Block List. Today, we are proud to be the first company to ship a scanning & compliance solution to have this advanced and unique functionality.

In our real world tests, once again we improv

– Pierre Chamberland
Pierre Chamberland is the Chief Energizing Officer at Messaging Architects.