Pivotal Veracity's Recap of the Authentication & Online Trust Association Summit
April 17 - 19, 2007
Boston, MA
Critical insights from this conference:
AOL, AIM
How does AOL's spam filtering work? AOL detailed their basic Anti-Spam architecture as a process flow and how it affects inbound email:Blacklist Filters: Your email will be checked against AOL’s internal blacklist.
Reputation Filters: AOL will review reputation metrics in the form of “abuse data” (spam complaints and spam traps).
Whitelist Filters: AOL will check if your from-address is in the recipient’s address book or if your IP and Domain are on any of their white lists.
From this point down you may bypass some or all of these filters based upon your whitelisting with AOL and if you’ve achieved, or not, the enhanced whitelist.
Volume Filters: If you’re not on AOL’s whitelist, your email will go through AOL's volume filters.
Content Filters: AOL applies its proprietary content filtering and Bayesian filtering rules.
Recipient Filters: Your email may pass a domain block, recipients personal Bayesian (defined by previous actions) rules or recipient’s personal filters
Your email will then be delivered, placed in the spam folder or blocked.
AOL claimed that the best way to improve deliverability is to setup a Feedback Loop with them to reduce the number of complaints they receive from their end users.
Plans exist to incorporate SPF & DKIM into the reputation model in the coming months. By late summer they hope to be actively using both as a means of authentication and reputation checking for connecting IPs as an added data point to help them determine the final disposition of email.
AOL recommends keeping content relevant and fresh. They suggest that you email only active users (subscribers or customers who have either opened, clicked or have an existing purchase history) as a best practice to achieving better deliverability.
COMCAST
How does Comcast's spam filtering work? Comcast detailed their basic Anti-Spam architecture as a process flow and how it affects inbound email:
Your email arrives at Comcast gateway at which time IP routing and reputation classification is established.
3rd Party Filters & Blacklists: Next your email will pass through the Brightmail Spam Filter and be checked against a variety of 3rd party blacklists such as the DNSBL.
Volume Filters: Comcast checks your email volume against traffic pattern models.
Content & Reputation Filters: Your email will pass through Comcast’s content and reputation (spam complaints, unknown user rates) filters.
Your email will then be delivered, placed in the junk folder or blocked.
91% of all inbound email to Comcast domains is flagged as spam.
Comcast is the largest broadband provider in the United States which makes them an easy and eager target for spammers.
Comcast monitors the quantity of hard bounces, specifically user unknowns, sent to their domains. They use this as a measure of list quality and maintenance. Sending active names (subscribers who have shown signs of open, click or purchases) is critical to ensure deliverability.
By the end of the year Comcast will adopt DKIM (Domain Keys Identified Mail) as an extension of their reputation check.
In addition, a Feedback Loop will be launched by the end of this year.
MSFT, HOTMAIL, WINDOWS LIVE MAIL
Ever wonder how MSFT's spam filtering works? Microsoft detailed their basic Anti-Spam architecture as a process flow and how it affects inbound email:
Blacklists & Volume Filters: Your IP and Domain are checked against MSFT's internal and 3rd party block lists. Throttling and-or traffic shaping also happens at this level; connecting MTAs can be completely blocked at this level.
3rd Party Filters: Your email passes through Brightmail’s spam filters.
Content Filters: Your email is then passed through Smart Screen (MSFT's proprietary content spam filter).
Whitelist Filters: Your IP and Domain are checked against Safe-lists and Sender Score.
Authentication check (SenderID): MSFT verifies your domain has authorized the mail to be sent from the sending IP Address (this method of authentication is known as SenderID).
Recipient Filters: Recipients' address books are reviewed to determine if your from-address has been setup in them.
A score is generated at this point which determines the final disposition of your mail (Inbox, Junk or Trash).
Hotmail receives 4.5 billion emails a day of which 90% are flagged as spam.
Statistics show that 35% of all spam today is image spam or messages comprised solely of one or multiple images.
SPF and SenderID records should utilize the hard fail mechanism “–all”. e.g. “v=spf1 mx –all” as opposed to neutrals (~all or ?all).
Every sub domain should have its own SPF and SenderID record.
Microsoft’s attack detection is geared to detect distributed bot-net attacks. IP’s sending a low volume of email at infrequent intervals will most likely be flagged suspect by this filter.
New IPs with no mailing history will have a harder time achieving inbox placement. Microsoft is relying heavily on mailing history as 1 aspect of reputation in order to determine where email will be placed.
According to Microsoft personnel, it’s better to email smaller volumes more often rather than larger volumes at infrequent intervals. Consistent mailings of 5-10K emails per day have been documented as performing well.
Microsoft doesn’t factor in the user’s address book the way that AOL relies on it as a determining factor for email placement. The address book is referenced late in the process, email can be blocked by other filters even if the user has added the sender to their address book and safe listed the domain.
Implications..
ISPs use a variety of different types of filters including 3rd party filters, blacklists, whitelists, reputation metrics, volume filters, and content filters. There are many reasons your message may not make it to the recipient's inbox and focusing on just one aspect of this equation (e.g. blacklists) is insufficient.
Reputation is measurable and is important but it is *not* something you can buy from a 3rd party nor do you need a 3rd party to have one! Reputation metrics consist of one or all three of the following depending on the ISP: Spam Complaints (the number of recipients complaining you are spamming them), Unknown User Bounce Rates (bad addresses), and Spam Traps (emails you send to email addresses harvested from the web or purchased). Managing Feedback Loops (spam complaints), good bounce management (removing bad addresses), and carefully governing your email acquisition methods are absolutely critical.
Content Filters are not dead and should never be ignored. Despite some companies implying that content does not matter, it does indeed as evidenced by each of these ISPs' stated use of 3rd party spam filters such as Brightmail that analyze content, Bayesian or adaptive learning filters that analyze content, and other proprietary content filters.
Blacklists matter and can be monitored. Verifying your IP address and URL domains do not get flagged on highly important blacklists still matters as all ISPs use them to some degree.
Volume filters are also at work at all ISPs. At AOL if you are registered on their whitelist, you can avoid the volume filter. Msft recommends mailing in lower volumes (stagger your mail) although this may not always be feasible for all mailers.
Authentication will become increasingly important and if you are not already authenticating with a combination of SPF, SenderID, and DKIM now is the time to get started as it will put you in good stead prior to the Holiday season. Whereas MSFT espouses SenderID, many others espouse a combination of SPF + DKIM or SenderID + DKIM. Yahoo is even requiring DKIM in order to sign up for their feedback loop (spam complaint reporting).
April 17 - 19, 2007
Boston, MA
Critical insights from this conference:
AOL, AIM
How does AOL's spam filtering work? AOL detailed their basic Anti-Spam architecture as a process flow and how it affects inbound email:
From this point down you may bypass some or all of these filters based upon your whitelisting with AOL and if you’ve achieved, or not, the enhanced whitelist.
COMCAST
How does Comcast's spam filtering work? Comcast detailed their basic Anti-Spam architecture as a process flow and how it affects inbound email:
MSFT, HOTMAIL, WINDOWS LIVE MAIL
Ever wonder how MSFT's spam filtering works? Microsoft detailed their basic Anti-Spam architecture as a process flow and how it affects inbound email:
Implications..
Upcoming Events
Speaking Opportunities
eec members have the opportunity to represent their company and the eec at many key industry events. For more information contact us.
Sponsorship Opportunities
Make a statement by sponsoring an eec event. eec event sponsors see a huge return on investment.
