We want our SPAM Back!

December 9, 2005 on 4:37 pm | In Insider View, Musings by Dallas Kashuba |

This may be the first time you’ve ever heard anyone ask for spam but that’s what I’m doing. We’re so committed to fighting spam originating on our servers that we’re willing to open up the floodgates and invite everyone to send us our spam back. Don’t start just forwarding us your spam willy-nilly, though. I’ll explain more about this later.

This whole idea started with AOL. No matter what you think about AOL, they are actually doing something real to reduce the amount of spam you receive every day. We still have a somewhat mixed relationship with them when it comes to email but at the end of the day I think the way they handle spam coming into their network is effective and I want the rest of the big email providers to do the same thing.

AOL has primarily put the control over what email is allowed to enter their network into the hands of their users. If their users report enough spam originating from a specific server or computer, AOL will temporarily block it, refusing all email for a relatively short period of time. DreamHost has been on the receiving end of that many times and it is annoying to a company in our position, but it has also motivated us to work on reducing spam. Not being able to send email to one of the largest blocks of email users in the world will have that effect!

When we first realized this was how AOL was doing things, we were honestly a bit appalled. “Oh, no! AOL users have the power to blacklist us?”

In addition to using user-powered email blacklists, AOL also sends us copies of every tagged spam they receive from our network. That’s what’s really good about their system and what I want everyone else to emulate. They send us the spams in a specific format to a specific address and we have developed an automated parsing system for them. As soon as we get the spams sent back to us we have cataloged them in a database for later analysis. We then use that data to determine where in our system spam is coming from and automatically stop new spam before it leaves our network, thereby reducing the amount of spam flowing over the Internet.

If the other major email providers (Gmail, Yahoo, Hotmail, etc) would all do this same thing DreamHost would be able to nearly eliminate all sources of spam on our network. If everyone sent back all spam they received to the originating network and every email provider processed their returned spam automatically like we do, spam itself could be nearly eliminated.

That, in short, is how AOL is reducing the amount of spam you receive little by little. That also explains why we’re asking for our own spam back. If you are a sizable email provider and you would like to join in the fun, contact me and we’ll get started. If you are an invidivual and would like to be involved, tell your email provider to get in touch with us.

22 Comments

  1. 1

    You’re welcomed to have my spam.

    Comment by Jennifer — December 9, 2005 #

  2. 2

    プロバイダにしかできないSPAM対策

    We want our SPAM Back!(DreamHost Blog)

    SPAMメールについての提言があります。

    現状のSPAMコメント、SPAMトラックバックの対処は、
     1.コメント、トラックバックを発見、削除(受動的)
     2.

    Trackback by しょうかいブログ — December 9, 2005 #

  3. 3

    thank you guys for the good press. working where i work is often a heartless and thankless job due to the rather rabid anti-fan base that we’ve accumulated over the years. however, its nice to open my rss client and read about good things that we’re doing and positive ways that we’re impacting people’s lives. thanks again guys. you made my night :)

    Comment by sungo — December 9, 2005 #

  4. 4

    Hi,

    One small point about this method (and extending it to the free account providers.) Doesn’t owning an AOL mail account correspond to having a paid account with them? And therefore keep an approximate one-to-one correspondence between voting blacklisters and individuals. However, extending this notion to the free providers where there might be a one-to-very-many correspondence between individuals and voting blacklisters (since any individual can sign up for tens or hundreds of free accounts) might lead to all sorts of fun and games.

    Comment by Eternal Mind — December 10, 2005 #

  5. 5

    for such things like this is that I am proud of being a Dreamhost customer. I always thought that global coordination was the key to stopping spam. But I doubt whether there are enough providers willing to make efforts to stop it. Spam is money, not only for spammers but also for security vendors that sell spam filters and for large network providers that rent temporary IP space for spammers.

    Comment by vinicius — December 10, 2005 #

  6. 6

    On a note related to “Eternal Mind”’s… What’s to stop people from gaming this system? We’ve now opened up the possibility for an email “DoS” of sorts… Script kiddie A decides he doesn’t like Web Store B, so he returns some fake SPAM to Web Store B’s ISP, getting their system to think Web Store B is SPAMing people, so they cut off his outbound email (or his account entirely, depending on the severity and policies, etc.). Now Web Store B is unable to continue doing business as usual because email plays such a prominent part in the business. It can no longer send order confirmations, lost password requests, or even register new accounts (because they require activation, using a link through… you guessed it, email).

    Maybe more details about what you guys do after receiving these kinds of notifications from AOL would help. What’s a ballpark number of “complaints” that have to come in before you’ll take action, and what action would you take? Are there any checks done to verify that this host was, in fact, sending out the SPAM?

    Sounds like a great idea in theory, but if the internet were an ideal place, we wouldn’t have people sending out SPAM in the first place…

    Comment by Chris Meller — December 10, 2005 #

  7. 7

    To Eternal Mind, aim.com mail is free these days (iirc).

    To Chris, both sides of this discussion are very aware of this possibility, i”m sure. Vindictive or just non-knowledgalbe users must always be factored into the equation. Also, good lines of communication and the ability for immediate escalation have to be maintained at all times. You’re very right in that there are very real concerns with setups like this. I trust both sides of this particular discussion to do what’s in the best interests of the users. In AOL’s case, anything to the contrary tends to end up in the washington post and the new york times, thus affecting the actual bottom line. Apart from goodwill, there’s monetary reasons to be helpful here :)

    Comment by sungo — December 10, 2005 #

  8. 8

    How about adding Certified Server Validation to stop spam too?

    http://itmanagement.earthweb.com/columns/executive_tech/article.php/3444571
    http://wiki.fastmail.fm/index.php/Certified_Server_Validation

    My primary email provider added that feature:
    http://blog.fastmail.fm/?p=522

    Comment by TB — December 10, 2005 #

  9. 9

    Wow. Amazing article! I’ve always despised AOL. In fact I consider AOL Instant Messenger to be filled with a spam-o-rama of insidious proportions.

    I agree with Eternal Mind and Chris, this could very easily be abused. They should definitely have some safeguards in place.

    Comment by Jonathan — December 10, 2005 #

  10. 10

    We consider each instance on a case by case basis. It depends on wether it is the customer themselves spamming, a broken script or otherwise exploited account being used to send spam, or a simple email address that is setup to forward to AOL.

    For customers directly spamming it doesn’t take much under our strict anti-spam policy.

    For exploited scripts we work with you to attempt, if we can, to identify the source of the email and either disable the script itself, or get you to upgrade it.

    For automatically forwarded spam we have somewhat of a conundrum. If someone sets up an email address here at DreamHost which forwards along to AOL, and then an email is tagged as spam we get dinged. The reasoning is we look complacent to spammers relaying through us, which makes sense. Basically we ask the person to stop pushing the “spam!” button in AOL, or disable the forwarding to AOL alltogether for that user.

    Comment by Kelly — December 12, 2005 #

  11. 11

    Thanks for the feedback, Kelly. I think that answered the majority of my questions about the basic setup. Unfortunately, this level of human intervention doesn’t really scale well, and eliminating the human element is really the key part: how do we get rid of as many of the the humans as possible in this equation, without severely compromising the effectiveness of the system itself?

    I’m sure you guys have put a good deal of thought into this already, and have probably considered a good number of alternatives for scaling this up. If you’ve had some kind of brilliant idea, I’d love to see another blog entry about it, as my 600+ SPAM messages a day is more than annoying at this point.

    Comment by Chris Meller — December 12, 2005 #

  12. 12

    I’d like to point out that this blog post is about eliminating spam coming FROM our servers and going off to other people and not spam coming into our servers and being delivered to you. That’s a whole other battle.

    This system I described is automated and relies on spam reports coming in from AOL. It may be possible for someone with a lot of free-time to ‘work’ the system and if it happens we’ll respond accordingly. There’s no reason to put any effort into combatting a problem that may never manifest itself.

    Comment by Dallas — December 12, 2005 #

  13. 13

    The post was also about other email providers doing the same thing and establishing a similar relationship to the one between Dreamhost and AOL. When it’s a 1-to-1 relationship, the chances of someone figuring out what’s going on and being able to successfully compromise the integrity of the system are slim to none. Once you start including more people, which means more information about the system is available, the chances of such things happening skyrocket.

    If you don’t want to talk about it, that’s fine. It was just a topic that drew my interest and I thought it might make for an interesting conversation. My mistake…

    Comment by Chris Meller — December 13, 2005 #

  14. 14

    Oh, I think I didn’t get my thoughts across well. I wasn’t trying to end the conversation or downplay your input. Sorry if I came off that way! It seemed like there was maybe some confusion about incoming versus outgoing spam. Maybe not!

    There would definitely be technical hurdles to prevent issues like what you described. The bigger hurdle would be actually convincing all of the big email providers to play nice together and go along with something like this, though. But anyway, to try to reduce the potential for exploit you could do some sort of cross-checking on reported spam and only consider it spam if it’s reported by more than one outside email provider and then only start blocking email once a specific spam is reported X number of times and that X could be separately adjusted for each provider reporting spam to balance out the reliability of their own systems.

    It seems like most of the proposals I’ve seen to reduce spam focus on reducing spam coming into a network and there doesn’t seem to be all that much focus on network companies working together to reduce the spam at the source networks themselves. I’d like to see more companies take more responsibility for that.

    Comment by Dallas — December 13, 2005 #

  15. 15

    I actually hate AOL for their spam. I had to stop using AOL for most e-mails, and now I have a blocker on all but e-mails of people and companies I know. I am constantly fishing friends of friends, et cetera, out of the spam bin for this.

    Now, you want good spam blocking, go to Gmail! If I happen to get a lot of spam, in a week or so, they’ve figured out how to block it all and send it ALL to the spam folder. I rarely get spam. When I do, it’s new and persistant, but like I said, gets completely blocked with time.

    Comment by DesignerElla — December 13, 2005 #

  16. 16

    What I would like to see is a database of all service providers email address for reporting spam/adward/spyware/viruses. This would allow us to send back the ip address of spam on web pages and the contents of those spamming emails. After all it is the service provider who has the real power of shutting these people down. They can terminate their accounts and/or notify the authorities which can result getting a warrant for the home address of these people.
    Silk

    Comment by Silk — December 15, 2005 #

  17. 17

    Try Gmail - it’s pretty good at fighting spam if you choose an email address that doesn’t have an english word in it (ie. first initial+last name@gmail.com).

    Comment by Stephen — December 16, 2005 #

  18. 18

    Sorry about the link above. It’s supposed to be last name @ gmail. com

    Comment by Stephen — December 16, 2005 #

  19. 19

    Thats is good and smart reason to ask for spam , in that case we all want the SPAM

    Comment by Design — December 26, 2005 #

  20. 20

    C it is new with you! Did not miss?
    I adore to communicate.

    Comment by Fina — December 27, 2005 #

  21. 21

    I wish there was a fuller discussion of how to assist / participate.

    Razor is gone.
    The new “Junk” email stuff seems to be this new anti-spam effort.

    One think though, the new “Junk” email stuff is not permitted on accounts that have the “catch-all” ability / account enabled.

    Because they catch “lots” of junk email.
    Well.
    Isn’t that a *highly* useful category to have?
    If only as a predictive / confirmatory indicator / telltale?

    Further, what about deliberate “HoneyPot” domains?
    Domains that have no purpose in life except to collect unsolicitied email?
    Virtually all email they would receive, other than from their registrar / web email host?) would almost certainly be spam.
    Similarly, “HoneyPot” email links posted on the web, deliberately placed / enabled for web-bot email harvesters to collect?

    Shutting off IP addresses from the email world for 4 hours or so at a whack is going to be uncomfortable, most especially if there isn’t some sort of “spam hold” type of status for users to become familiar with.

    Comment by nekote — January 19, 2006 #

  22. 22

    Yahoo recently asked me to read some distorted letters and type them in a window BEFORE they would accept an email I tried to send. They told me that this was one of the means of preventing spam from being sent from their hosts. They don’t do it frequently, but every so often.

    I think that Dreamhost has to be proactive in that way. You have to implement ways to make sure that your clients are not spammers. Waiting for others to tell you that is kind of too late. It’s a good check on your system, but it’s only after the fact.

    PREVENT!

    Comment by Jurek — February 1, 2006 #

Sorry, the comment form is closed at this time.

Powered by WordPress. Pool theme by Borja Fernandez, modified by DreamHost.
Entries and comments feeds. ^Top^