It's pretty unlikely that employees wished this holiday season for even more email and spam messages to be waiting for them in their inboxes each morning. However, the reality is that organizations - across all industries - have experienced a dramatic increase in inbound email over the past calendar year, much of which is spam, and inbound email volumes will likely continue to increase at a rapid pace during 2007.
In a recent study of trends across our enterprise customer base, Proofpoint found that the volume of inbound email to large corporations, universities and government organizations grew at a staggering rate during the past 12 months. Many companies saw inbound email volumes grow between 200 and 300 percent, with some experiencing as much as a 700-percent increase in email volume during this time. And in just the past few weeks, many organizations have seen double-digit percentage increases in inbound email volume.Why more spam? Why now?
As enterprise anti-spam solutions have increased in effectiveness, spammers have developed new spamming techniques - making spam much more difficult to detect - while greatly upping the overall volume of spam being sent. The following advances have helped spammers evade less sophisticated spam filters, thus increasing the return on investment for spam attacks:
· Image-based spam: Many spam filters are challenged by the latest spam trend where images are used to deliver the spam "payload." Graphics are embedded in the spam message, with spam content inside of graphics, making it difficult for many spam filters to identify. While image-based spam is not new, it's become much more popular with spammers who are using advanced techniques (such as making small variations in each image to avoid signature-based detection, hiding payload images in animated GIFs and using interlaced images that are resistant to optical character recognition techniques). Proofpoint estimates that image-based spam now constitutes 15 to 20 percent of total spam volume.
· Expansion of botnets: Botnets, networks of personal computers controlled remotely by unauthorized users to send masses of spam without the system owner's knowledge, account for as much as 75 percent of today's spam messages. This problem has been exacerbated by recent virus attacks, such as the Warezov mass-mailing worm, which often drop a botnet client on infected machines.
· Continuing use of text obfuscation: Some spam filters struggle to identify obfuscated words (which can be a strong indicator of spam) and to differentiate intentional obfuscations from unintentional obfuscations (such as spelling errors). Common techniques used by spammers include the replacing of characters with similar characters (i.e.,Viagra with Vi@gra) and eliminating vowels (i.e., Vicodin with Vicodn).
· Foreign-language spam: Enterprises around the globe have seen a substantial increase in spam in multi-byte languages such as Japanese and Chinese and many filters are not yet capable of handling these messages.
· Spear phishing: An evolution of traditional phishing techniques, spammers leverage a recognized brand that is affiliated with an organization (i.e., an email would appear to originate from a company's outsourced payroll company, as employees are used to receiving and opening email correspondence from that particular organization).
· Hash busting: Aimed at confusing spam filters that use Bayesian analysis techniques (calculating the probability that an email is spam based on message contents), spammers include seemingly nonsensical words in order to make the email appear to be personal correspondence.Moore's Law for spam filters?
New spamming techniques and a massive increase in the sheer volume of spam being sent, combined with the overall increase in legitimate email messages, has had a predictable but unwelcome result. Many email users who had become accustomed to spam-free inboxes find themselves now receiving a noticeable and often annoying amount of spam, which can lead to an increase in end-user complaints and helpdesk calls. In addition, email administrators may find themselves spending more and more time trying to stem the flow of spam.
In some cases, anti-spam solutions are unable to keep up with spammer sophistication, but in others, the spam filter effectiveness has remained constant - or even improved -- but the higher overall volume of email results in an increase in the absolute number of spam messages making it through the filter.
Spam filter effectiveness is typically determined by looking at the number of spam messages that are blocked or captured versus the number of spam messages that evade the filter (sometimes referred to as "false negatives"). Also important is the number of "false positives" - legitimate email messages that are incorrectly categorized as spam. Effectiveness is the most common metric used throughout the industry to compare the quality of anti-spam solutions.
According to security analysts at Gartner, today's best-of-breed anti-spam solutions should be at least 95-percent effective and require administrators to spend a maximum of one to two hours per week managing every 5,000 mailboxes. But even this seemingly high level of effectiveness isn't enough to keep users from seeing an absolute increase in spam. For example, in a company with an annual tripling of email volume, a spam filter that was 95-percent effective in November 2005 would have to be 98.3-percent effective today just to keep the total number of "false negative" spam messages at a constant level.This has created a new kind of "Moore's Law" for spam filters, in which filters greatly increase their effectiveness every year just to stay "on par" with past performance. (Moore's Law is the storied "law" in the semiconductor industry stating that processing power doubles every 18 months.) This dynamic should spur companies to push for ever-higher levels of spam filter effectiveness and innovative new spam detection techniques from vendors, as filters will need to be well over 99-percent effective by 2008 if current email trends continue.
Boosting anti-spam effectiveness
The sheer volume of email - both legitimate and unsolicited - has led to a perception crisis around the effectiveness of anti-spam products. Some organizations that have seen a jump in spam messages getting through to end-users are quick to blame the anti-spam filter, assuming that the product is not working as well as before, when in fact this might not be the case. While not all spam filters are created equally, organizations that are unsatisfied with their current level of spam protection (or concerned about these trends) should investigate the following:
· Effectiveness benchmark: Gartner says that IT organizations should benchmark their anti-spam efforts over time against the metrics above (at least 95 percent effective and administrative requirements of one to two hours per 5,000 mailboxes) and make adjustments or investigate new solutions if you find your results are outside of these best-practice ranges.
· End-user controls and education: Educate employees about the continuing rise in email volume and offer more aggressive levels of spam policies for users. Activate end-user digest or quarantine features to demonstrate to users how many messages are currently being blocked.
· Predictive technology: Mounting an effective defense against spam requires detection techniques that can evolve as quickly as the attacks themselves. How is your vendor addressing the most challenging forms of spam; does the product anticipate attacks and predict components of spam campaigns? Does your anti-spam vendor provide continually provide timely updates to address new forms of spam?
· Overall effectiveness rating: Some organizations fall prey to basing filter effectiveness on select users' experiences - typically individuals who are most vocal about perceived changes in filter performance - rather than taking a big picture look at effectiveness across the organization.
· Use today's anti-spam budget to invest in messaging security: Faced with the issues described above, many organizations (especially those with first-generation anti-spam solutions) will be taking a fresh look at alternative solutions. If you're allocating budget for improving email security, use this opportunity to investigate solutions that offer additional messaging security features -- such as outbound content security and multi-protocol content scanning capabilities - in addition to spam blocking.
· Plan for increased capacity: Review your planning assumptions about the hardware resources you've allocated to messaging security. If you deployed a solution more than 12 months ago, chances are that your systems are approaching their maximum capacity. Email volumes will continue to increase - work with your vendor to ensure that your solution has enough day-to-day capacity and headroom to handle today's massive spam attacks.Enterprise concerns about spam filter effectiveness, whether real or perceived, are a legitimate issue that require attention. As spammers gain sophistication and email volumes continue to rise, organizations need to evaluate their existing anti-spam solutions, determine current levels of effectiveness, and adjust accordingly in order to ensure that their anti-spam solution of choice is just as effective at identifying spam tomorrow as it is today.
-Sandra Vaughan is vice president of products and marketing for Proofpoint.
