New Dream ‘Alloween!

No talking, just pictures!

Farewell Charis

Today most of the DH crew went out to the East office and had a farewell lunch for Charis. She’s off to be a full time mom. Here is a quick video from the lunch table.

The Secret of Our Succe$s

Get out your pencil$ and notepad$…

Recently Joel $polsky put up an article about how to pick what features to include in $oftware.

I read it accidentally, and it turns out he almost exactly describes how our $uggestions Area works!

We one-up his method though… rather than just having employees “buy” features they like, we put them up there for our entire customer base to vote on!

In that way, they (you!) do the most important thing to our entire company’s survival for us… you tell us which features we should work on ($ubversion) and which features we shouldn’t (Donate idle CPU time to BOINC).

(Oh, you want to know the most-suggested suggestion that we will never implement? I’ll tell you anyway: Allow seeing how many votes suggestions have. HA! I don’t THINK so! The only way we can be sure that we’re getting an accurate view of the wisdom of crowds is if everybody’s votes are independent. The last thing we want is people voting for something just because it’s ALREADY popular, or NOT voting on something just because nobody else has yet.)

But you know, just having customer-votable suggestions does not instantaneously imply succe$s (especially not the $ part)… there’s one more critical missing element. And that is the true “Secret of our Succe$s”..

HA! You think I would tell you all our secret$?!

You’re right. I will. Here it is:

We then actually implement the features!

But.. how we actually get done the things you know you need to do… THAT is our real $ecret!

HA! You think I would tell you all our secret$?!

Okay fine, I will.. one last time.

It’s the newsletter.

You see, just under 8 years ago a customer wrote in to me (yep, yours truly, the founding member of the Happy DreamHost $upport Team) suggesting we have a newsletter, because she didn’t get much email and would like to know what was going on in the company and things like that.

Well.. at the time we didn’t much need a newsletter… since there wasn’t much going on in the company.

Fortunately for DreamHost, my ego grabbed me by the throat, wrestled me to the ground, and said “Jo$h, write this newsletter. YOU WILL BE AN INTERNET GOD!” And that’s how the newsletter was born.

I don’t think I’ve ever told that story before.

But what does the newsletter have to do with us implementing new features, you ask? Well, since I decided to make it a monthly publication, each and every month I began needing what we in the business call “newsletter fodder.”

So now, around the 28th of each month (the 27th in February) I realize once again we’ve done nothing all month, and so frantically get everybody to $lap together some half-baked features from the top of our suggestion list, copy them live, and dump them in the newsletter!

We then generally spend the next month (up until the 28th) fixing those features.

$urprisingly, it actually kind of works. It sometimes makes for some ugly bugs, but hey, we’re not making pacemakers or space shuttles here. We’re providing $7.95/month web hosting. Not to mention, the uglier the bug, the quicker the fix.. and thank heaven we’re entirely web-based: bug fixes are “distributed” to all our customers in about 2 minutes.

So there it is.. the only thing keeping DreamHost competitive in the fast-paced world of Cheap Web Hosting… is Josh’s all-encompasing fear of an empty newsletter.

And you can take that to the bank$.

More Fun With Spam!

Well, so, Josh didn’t get his wish about the video Airport Express, but I still think the new iPod video stuff is neat (especially the part about downloading TV shows). But we’ve been a bit too busy today to debate the finer points.

The thing keeping us busiest this morning was a huge influx of referer spam. This is a really dumb problem. The spammers just want to have a high ranking on our customers’ pages that list referers, like this one:

This not only gets their URLs in front of anybody who views that page, but they also believe that it’ll help their Google ranking. Which is dumb because it’s easy for Google to spot and work around.

And the absolute DUMBEST thing they do is post this referer spam at a high enough rate that it actually overwhelms the webserver. Which creates obvious headaches for us and makes us do something to stop them. We usually can just isolate individual cases and block the attacking IPs, but this morning was so huge and pernacious and widespread we sort of had to solve the referer-spam problem itself, just so these jerks will leave us alone.

So we’ve stopped most of it with mod_security and will keep our eyes on things.

The biggest lesson is that if you write or run any sort of bloggy software that has a page for the top referers, get rid of it! It’s just asking for abuse from unsavory types!

Video Killed the Audio Star

So… apparently after the iPod Nano debut on September 7th, Apple has “One more thing…” to unveil next week.

Rumor is, it’s going to be a video iPod.

Well, it may be. Or is may not be. Who cares? I’m not here to speculate about the rumors. Oh, no..

I’m here to talk about what I would be unveiling on October 12th if I were Steve Jobs.

I’d be releasing a VIDEO Airport Express!

I’d call it “vTunes” (Honestly? I’d probably call it “JoshTunes”) and it’d let you wirelessly stream HD-quality video from your mac/pc/desktop/laptop via magic (or something faster) to your TV! (FYI, the current Airport Express is a doo-hickey that lets you stream music from iTunes on your computer to your home stereo(s) over 802.11G)

But this is probably all just a crazy coked-up pipe dream of mine.

OR IS IT????!!!!!!????!?!?!!!!!

Apple already claims their new H.264 video codec can do FULL 1920×1080 24p HDTV in just 7-8Mbps!

And as we all know, their Airport Express already handles speeds up to 54Mbs!

(Of course, that’s just the theoretical maximum. Just like the theoretical maximum height for a human is 8′ 11.1″. But we can’t ALL be Robert Wadlow, can we? Similarly, I’ve found it’s usually a good idea to take the theoretical maximum and divide by EIGHT to get a rough idea of the real speeds you’ll experience. Fortunately, that puts us really close to their 7-8Mbps range!)

We also know that iTunes has been able to play video since version 4.8.

Add it all up, and I’d say there’ s about a 98.6% chance that “vTunes” will be announced before or during January 2006. (Maybe as soon as next week!)

Personally, I think they’d sell like hotcakes, be the biggest thing since sliced bread, and have their cake and eat it too! Unlike on an iPod, people seem to already like watching video on their televisions. People also seem to already like the convenience and flexibility digital media offers. And people already seem to think Apple is the gall’s balls.

Here’s how it’ll work:

1. People download movies/tv shows/music video/flash animations/porn from the vTunes Video Store. Alternately, they rip full DVDs to their computer using vTunes (or maybe some other software like DVD Shrink).

2. They watch any video on their hard drive or directly off the Internet(!!) at any time on their 102″ plasma without going to BlockBuster, opening their mailbox, or even standing up!

They won’t have to worry about their DVDs getting scratched, or lost, or disorganized, and they can do things like suck movie info right off IMDB for their whole collection and sort by things like best movie or shortest running time or most Josh Hartnetts.

And it’ll work. And it’ll be awesome. And I know it for sure.

Because I’ve already got one!

You see, I’ve got a modded Xbox hooked up to my TV, and as of September 12th Xbox Media Center finally has full native DVD menu support! Which means I can rip my DVDs to my home computer and then play them wirelessly over the network on my TV via XBMC on my Xbox.

Further still, XBMC connects to IMDB via the Xbox’s ethernet port and lets you do exactly the sort of things I was “imagining” above! XBMC even has python script plugins which let you do things like watch the videos at ebaumsworld or listen to podcasts directly on your Xbox. Just like that!

It’s a weird feeling, effectively creating “Internet channels” on your TV.

But it’s a weird feeling that feels good.

Just last night I watched Superman III via windows file sharing from a full-sized DVD rip (no shnazzy H.264 codecs or nuttin’) on my laptop via 802.11G. And it really worked.

Superman III is funny because Richard Pryor is a bum who reads a little matchbook that says “Become a Computer Programmer” and so he does it and he’s really good at it and he falls into working for a bad guy who’s just like every other boss a programmer has ever had asking him to do all these miracle technology things with a computer that he has no idea how to do himself… except his requests are like “Write a program to send all the oil tankers in the world to the middle of the Atlantic” and “Destroy Columbia.”

That may have been a funny way to end this post.

But I’m serious about the vTunes stuff.

And I’m right about it! You’ll see. All of you…

I love the smell of twisted metal in the morning!

Someone can’t park!

Driving in to work today we were all greeted by this lovely scene in the parking garage.

Looks like someone zigged when they should have zagged! Hoo-hoo!

Credit for the photos goes to Scott from our tech support team!

Silly Bugs!

One little problem that cropped up since we pushed our swanky new web panel live was somehow domains were getting into our database with CAPITAL letters in them!

This causes some problems with adding email addresses, dns, domain registration and things like that in our system that assume a domain will always be all lowercase (since domain names are really case-insensitive).

At first glance, I couldn’t see how the domains were getting into the system like that with this check RIGHT THERE.

Can you?

The Real Damage From Spam

Anyone who has used email in the last couple of years has probably had to deal with spam at least a little. Most people probably consider it an inconvenience that has to be endured as part of using email. Anyone who has had the same email address continuously for more than a few years probably see it as more than just an inconvenience as you probably receive a couple hundred spam messages a day. You also almost definitely use some sort of filtering system as well, though. I know I personally would be literally unable to deal with my email if I had no spam filtering. I’m probably a bit of an extreme case, but my junk mail filters kept about 850 spam emails from hitting my inbox just yesterday.

The annoyance of dealing with keeping your inbox clean is only a very small part of the spam picture, though. Most of the real pain is felt by companies like us: web hosts, email hosts, and network providers. Spam and its associated problems is something we deal with on a daily basis and many days it is one of our top issues.

Our customers expect us to provide them with a service that’s as hassle-free as possible and that includes the amount of spam they receive in their inbox. The job of deciding what incoming email is spam is difficult because each email recipient probably has a slightly different opinion on the matter. On top of that, a single falsely rejected message is a problem so that must be minimized. To that end, we use a relatively conservative blocklist on all of our incoming email servers. Even with a conservative blocklist like that one, our incoming email servers block about 60% of all incoming email as spam.

Even with a blocklist, a lot of spam still gets through and into inboxes. For instance, my 850 spams from yesterday were all after the 60% that was rejected at the email server level. Following that logic I may have actually been sent a whopping 2125 spams just yesterday! To help catch the rest of that spam, we also have our junkmail filter service. We still have the junkmail filter set to off by default and most of you have not yet enabled it. We don’t know for sure, but we have estimated that it may take more servers to scan all of the email than it takes to deliver all of it to your computers. Right now, we have 3 powerful servers scanning incoming email and we will continue to add more servers as more of you start filtering your email. We’re hoping our estimates are wrong because that would mean it would more than double our cost to provide email to you. We won’t charge you extra for that, of course, so we will be bearing the entire cost.

And that’s not the half of it!

The other side of this coin is spam originating from our servers. As spam filters have become more aggressive and the legal system has begun to tackle the problem of spam, spammers have increasingly been going underground and essentially stealing server and network resources from companies like us to send out their emails. Our own customers do occasionally goof up and send spam, but the majority of the spam comes from hijacked accounts accessed through security exploits in software installed under your websites. We have notified our customers repeatedly about known software security holes, but many of them still do not upgrade. That puts us in a sticky situation as we do our best to never take down a website unless it is absolutely necessary. We have to choose between keeping the website up and running and potentially allowing more spam to leave our network. We have historically risked it and left the website up, but that is fast becoming an option we cannot take.

When a certain amount of email deemed to be spam leaves one of our servers other email providers sometimes decide to begin blocking all email coming from the server. As they are also trying to provide a hassle-free email experience for their users that move makes sense in many cases. It gives us time to determine the source of the problem and clear out the spam so it is not delivered to anybody. That would all be ok, except for the fact that many email providers then refuse to remove the block. We have a very strong anti-spam policy and are very quick to respond to any and all complaints we receive about a spam originating from our system and yet still we are blocked. Some of them tell us they won’t unblock us unless their own customers complain to them about it. That doesn’t seem to be a very customer-focused way to do business to me. Forcing your customers to come to you and complain before you will provide them with a service they are already paying for (email to and from anybody!) is just plain crazy. That’s how bad spam has made things for companies like us and them!

The only option available to us now is to start limiting our own users and their legitimate use of email to protect our network from being blocked by trigger-happy email providers. We have long believed in leaving our system as open as possible so people can use our services in the ways that best meet their own needs, and it is always hard for us to make decisions contrary to that philosophy. Things have changed quite a lot since the days of the happy-go-lucky Internet of yore. There is some interesting looking technology on the horizon that may help the situation but it will be a long time still before anything like that will be truly effective. For now, we’ll just stay in the trenches fighting the noble fight.