Why web hosting is hard.

August 4, 2005 on 11:22 pm | In Musings by Josh Jones | 38 Comments

Nate tries programming.

I am so jealous of Ebay.

Ebay is so easy.

Web hosting is hard!

At DreamHost, we average about 750 incoming support messages a day.

We also run DreamBook, a free online guestbook service with about 20 times the users as DreamHost, and it gets about one message a week. That’s five orders of magnitude less!

Why so few messages to DreamBook?

Well, apart from the fact that we make it kind of hard to find the support address, DreamBook is an entirely web-based application. That means we have full control over what our users are allowed to do, the interface they use to do it, and how the application interacts with our hardware configuration.

Since everything a user might possibly do with DreamBook was “invented here,” it’s always been pretty straightforward for us to fix problems, add requested features, make things easier to use, and generally keep things humming along. Every once in a while there’s a blip and some problems come up, but if it’s worth it we fix it, and then it never bothers us again. It’s nice to be an “Ebay” or a “SalesForce.com”, and to run just one web application (it’d also be nice to be making a fridgeload of money off it.)

Why so many messages to DreamHost?

Because web hosting is hard! It’s hard for customers and it’s hard for hosts. The web hosting business is essentially just renting out general computers to every Joe, Dave, and Sandy on the web for $7.95/month. Then, these shmoes and shmo-ettes try to install and run every application under the sun (not just Java).. not the least of which is their own custom stuff. I won’t even get into the fact that a good percentage of your own users are actively trying to crack the system!

Surprisingly often, these applications just don’t quite work. Or just don’t work with each other. Or just not with our setup. And since they’re not things “invented here”, we often have no idea what’s wrong or how to fix it. But our customers aren’t the half of it.. even our OWN system is made up of tons of different software packages we have very little control of. There’s MySQL, Squirrelmail, Postfix, Courier, Apache, Debian, PHP, Ruby (on Rails), Mailman, Jabber, SSL, EPP/RRP (domain registration protocols), Cybercash, Perl, Python, and all the various modules and libraries everybody needs installed for each of these.

Not to be brash, but the parts of the DreamHost system we have full control of generally work pretty well these days! The billing stuff, our web panel, our announcement lists, our rewards system, our one-click installs, our support system, all that garbage! Everything we’ve built from the ground up is pretty well understood and easily maintainable by us. And, like DreamBook, when bugs do pop up we can splat them quickly because they’re our bugs!

However, if pwcheck keeps dying on our mail machines because we’ve got 150,000 mailboxes in a cluster, there’s not much we can do other than rewriting the thing from scratch. As you can see above, Nate actually did re-write pwcheck from scratch.. that’s how critical it is to have a good understanding of components vital to your system!

As far as Internet businesses go, web hosting is hard.

(And as far as IDEs go, tablets suck.)

38 Responses to “Why web hosting is hard.”

  1. LittleZephyr Says:
    August 5th, 2005 at 12:18 am

    “I won’t even get into the fact that a good percentage of your own users are actively trying to subvert your business!”

    Could you get into this? I’m kind of interested in what you mean here.

    ~ Happy ( Non-Subverting ) DreamHost Customer

  2. josh Says:
    August 5th, 2005 at 12:54 am

    Ah, I re-phrased that now. I just meant fraudulent accounts: spammers, phishers, warez, crackers, irc bots, ddosers, and their ILK!

  3. Robert Waugh Says:
    August 5th, 2005 at 1:32 am

    Hey, quit trying to humanize yourselves. What an obvious ploy. Let’s all pretend that live human beings populate the tech support domain, ha ha ha. As far as I can tell, you’re either sophisticated drones or possibly elves.

  4. andrew lucking Says:
    August 5th, 2005 at 5:31 am

    Just last week you were bragging ’bout all the bling you guys get and now you’re whining about how tough your gig is? ;-)

    Seriously, it’s awesome to get an insider look at what you guys do. Pros and cons.

    A.

  5. Jim in the Buff Says:
    August 5th, 2005 at 5:43 am

    I like the new Dreamhost Blog, and I think that a lot of other users will like it too, especially if you tone down the profanity and maybe be careful about referring to your own scripts as garbage. I’m as in tune with the f-word as the next guy, and, at the same time, I’m sure there are other users who aren’t, and who might take offense at phrases like “Where the f— is my database?” Nevertheless, I like the blog a lot. I’ve never used a web host that was so transparent and open with its customers, and it’s refreshing. Other web hosts I’ve used act like they’re huddled up inside some crystal palace, afraid to even look out the windows.

  6. ET Says:
    August 5th, 2005 at 7:00 am

    I like the Apple keyboard in the background. You guys should take over all the “old” Virginia Tech G5’s and increase your processing power give them out to customers.

  7. jr Says:
    August 5th, 2005 at 9:14 am

    Well, I know how hard it is, how depressing some of those support requests can be, and how hard it is to keep stuff moving, particularly when you have no control over it.

    I’m damned impressed with the job you do, and exceedingly forgiving when things don’t work quite right or you don’t have the newest coolest version of OpenWidget-9000.

    Nice thing is, I’m betting that the vast majority of your soon to be world conquoring army of zealots feel the same way.

  8. Jack McCormack Says:
    August 5th, 2005 at 10:39 am

    Stop moaning and give me all your money

  9. Micah Says:
    August 5th, 2005 at 3:04 pm

    As a new customer, I think the service has been outstanding. I usually get a quick reply from tech support, and understand that replying could take up to a full day or two now knowing that you receive so many support e-mails everyday. Outstanding job!

    I have a few ideas, however, that might help you out a little.

    1. You should request for volunteers out of the dreamhost community for help. You could easily promote them to moderators of the forums or wiki so that they can answer user questions rather than relying on the few of you (all of you?) who answer all of those support e-mails.

    2. Obviously, most problems are user error. People don’t know what they’re doing, and they tend to rely on tech support for all of their answers even if the answer is just a few clicks away on the knowledgebase, wiki or forums. Thus, you need to introduce a better promotion of these areas. They aren’t obvious choices, and getting to the wiki usually takes me 4-5 steps from my control panel. I think these should be much more prominent in the control panel, even more so than the support button. Make it blatently obvious that these are there and usually contain all of the answers to the questions people have.

    3. Make the wiki and forums more important than the knowledgebase because not only can it be immediate feedback because of replies and comments, it’s just as easy to use and find answers.

    4. Maybe this isn’t a great idea, but the live customer support system: can you have some volunteers help ya out? Have them use the system just like you do, but obviously don’t give them as many privileges or rights as an admin.

    Maybe you’ve thought of all of these ideas already, but I think that relying on more of your customers for help would be a great use of that resource. I’m sure there are people who enjoy helping others and would happily commit to this kind of opportunity.

  10. Sam Says:
    August 5th, 2005 at 3:05 pm

    I dig this blogg too! That pic above is from the SIGGRAPH show in Los Angeles which was this week. thats the new Wacom tablet screen- its whay trick.

  11. Alex Says:
    August 5th, 2005 at 4:49 pm

    as I can see what above? the wacom tablet? what does that have to do with nate re-writing pwcheck?

    (you guys gonna GPL it for more publicity?)

  12. josh Says:
    August 5th, 2005 at 5:33 pm

    Heh, Nate rewriting pwcheck on the Wacom (pronounced “WACK im”) tablet was just a jokey joke! That was indeed at Siggraph!

    And yeah, we intend to make the Wiki more prominent eventually, as it gets more verbose and populated.. and probably eventually phase out the Kbase even. Still, the Kbase has more stuff than the wiki though. So we wait!

    Stay tuned for “Why web hosting is easy.” next week!

  13. ricky Says:
    August 5th, 2005 at 6:31 pm

    aaah the true joy of running a business, having something decent to bitch about :)

    Great blog Josh

  14. John Doe Says:
    August 5th, 2005 at 7:16 pm

    You guys have a really dense Knowledge Base. However, you message board stinks. It is just plain ugly. I suggest that you guys convert the Feedback feature in Knowledge base section into a full fledge forum. Also, you should read some of the comments in KB and update the KB accordingly. It will reduce your support requests tremendously.

  15. Will Says:
    August 5th, 2005 at 7:29 pm

    Since we are OT anyway, I would love to see new forum engine used along with a DreamHost redesign and promotion like on the top menu-bar. It really does answer and a lot of user suggestions and it does serve an important roll. Have community helpers and then your Honchos/Employees (thnx to Jeff, Dallas, Will – still drops by, Josh, Nate for making an effort to post).

    Why it would benefit DreamHost, because forums and really happy/helpful user-base really does say a lot of about a hosting company. :-)

  16. David Quinn Says:
    August 6th, 2005 at 8:21 am

    I am a new Dreamhost Customer (literally one week ago today I signed up). I think you guys have done a fantastic job, and pretty well made everything self-explanatory. We appreciate all the hard work you put into this for us. This is is my first time with a real hosting company (I have bought accounts at Yahoo, and Typepad, but they were premade), and I appreciate how simple and fast you guys have made the process.

  17. Jim in the Buff Says:
    August 6th, 2005 at 12:11 pm

    Hey, have you ever seen the message board system called vBulletin? That’s the one used by herogames.com and I think it’s the coolest message board system that I’ve seen, and I’ve seen at least three. No, wait, four. I’ve seen four. Also, I’d love to see Coppermine Photo Gallery included as a One-Click Install.

  18. Kelly Says:
    August 6th, 2005 at 5:45 pm

    To respond to Jim in the Buff:

    We cannot include vBulletin because it is not FOSS (Free Open Source Software), and paying for it would likely remove any semblance of one-clicky-ness the oneclick installer has.

    Otherwise, head over to the user driven suggestions about the Coppermine Oneclick:

    https://panel.dreamhost.com/index.cgi?tree=home.sugg&

  19. jarek Says:
    August 7th, 2005 at 7:33 am

    Hey guys, here’s some math correction for you:
    750 msgs/day x 7 days = 5250 msgs/week. Compared to 1 msg/week this is THREE orders of magnitude more :) You probably wouldn’t survive five orders of magnitude more messages per week :D

    cheers,
    jarek

  20. josh Says:
    August 7th, 2005 at 11:47 am

    PER user I meant.. so multiply 5250 by 20! :)

  21. Deamos Says:
    August 7th, 2005 at 2:31 pm

    I worked for over six years in the web hosting industry and can atest to the suckiness of it. I am now only peripherally involved in it in the company I am working for now.

    However you guys do an outstanding job at it. Every place has its bumps but you all seem to weather them pretty well.

    Better you than me! I burnt out after a while. :)

  22. Anonymous Says:
    August 7th, 2005 at 7:13 pm

    following the links on the bottom of the page – “Valid XHTML and CSS.” this page validates as neither?

  23. josh Says:
    August 7th, 2005 at 10:32 pm

    Ha! It was the part I’d customized. I guess I’m just not a valid XHTML/CSS generator!

  24. jarek Says:
    August 8th, 2005 at 5:50 am

    Ups. Well, hell I’m impressed now. Gee, that’s one hundred thousand messages per week! Unbelievable.

    Hats off guys!

  25. josh Says:
    August 8th, 2005 at 8:07 am

    Heh, er no.. I didn’t mean we get 750 questions per dreamhost user per day! :) Just that the questions per dreamhost user is 100,000x the questions per dreambook user! There.

  26. mybeau Says:
    August 8th, 2005 at 8:21 am

    Hi there
    Hosting is a pain, and it’s a difficult job but I’m sure you love it. As well as we, customers, do love Dreamhost :-)

    But you’re right to complain. Griping and grnarling ease things… And after all, again we customers, do complain too..
    As I can’t possibly write that in Support or Troubleshooting, here I go… I like DH very much. Thanks for your efforts guys and thanks for this blog too.
    Too bad you won’t read mine :-)

  27. Brooks Says:
    August 8th, 2005 at 10:02 am

    I want that tablet!!!! Oh, and looking forward to “Web Hosting is Easy.”

  28. DreamHost Blog » Why web hosting is easy. Says:
    August 8th, 2005 at 9:33 pm

    [...] Even if web hosting is one of the hardest Internet businesses, it’s still pretty darn easy compared to the rest of the business world. [...]

  29. Eirik Says:
    August 9th, 2005 at 8:47 am

    But you’re right to complain. Griping and grnarling ease things… And after all, again we customers, do complain too..

    Complain? I never complain… ;)

  30. DJ Coffman Says:
    August 9th, 2005 at 3:24 pm

    You guys fucking rock. I just wanted to say it again publicly. Back in May or April, I got slashdotted with the Darth Vader blog traffic, and it was killing the CPU on the server— I had no idea what the hell CPU usage was, but apparently, I was killing a server…. then followed up by that, I got linked at Penny Arcade and that was even MORE traffic…. soon I was afraid as it was suggested to me, I would need to get like a 99 dollar account, which i just can’t afford for our little webcomic effort…

    But Dreamhost worked with us, put us on the temp server, and we brought our CPU usage down with a few tips from the techs— damn, if there was an award or a cyber hug or high five I could send, I would.

    Now it’s awesome reading some of the behind the scenes things going down.

    Keep up the good work dreamers!

  31. cednie Says:
    August 12th, 2005 at 2:48 pm

    Nice blog! I like it. Hey, is there a way you can link to general system-wide messages (emails) that you sent out on the front page? Or put them in the blog? Just a suggestion.

    Happy Dreamhost customer for years.

  32. killjoy Says:
    August 17th, 2005 at 2:52 am

    Heh, I’m probably the worst for uploading nefarious PHP applications to try them out. :x “This al Qaeda photoblog.php.exe.bat.com script will be PERFECT for what I’m doing!”

    I’m not a nefarious criminal though, I’m just your average web… person, learning as I go, and a simple, humble, and quiet webmaster, just trying to do my own thing.

    So far I’ve had numerous web projects hosted with two other hosts, and dreamhost is hands down, the best, the tech-support is second to none, the knowledge base is fantastic, and I always recommend dreamhost to anyone even thinking about hosting.

    You guys do indeed roxx0r, I appreciate doing business with you guys. ^.^

    Cheers!

  33. -= nonecknoel.com =- Says:
    August 27th, 2005 at 7:31 pm

    the times they are a changin

    there are times in this world when you think, “humm i wonder how this is going to play out?”

    for the 24 hours leading up to announcement of google chat, i too was wondering WHY are they doing this and HOW will this play out. i hopped on the

  34. Dmitriy Says:
    December 21st, 2005 at 7:32 am

    I want to join. How I can?

  35. DreamHost Blog » Why Web Hosting is Here to Stay Says:
    May 31st, 2006 at 9:14 pm

    [...] But at least I don’t have to worry about Google putting us out of business! Why? Because Web Hosting is Hard, and Google don’t want it! Nah, they don’t want it! They don’t want to give customers what is essentially a full-featured computer to run.. they want to give customers cool ajax-y websites that are easy to use and easy to mine for data. [...]

  36. DreamHost Blog » Why web hosting is fun. Says:
    July 11th, 2006 at 2:31 pm

    [...] Josh has previously told you why web hosting is hard and why web hosting is easy, and now you’ll find out why it’s fun, too. [...]

  37. ripper Says:
    October 5th, 2006 at 10:17 pm

    Oh, come on… webhosting now is easy try 1990 when I had to have 3 air conditions just to cool my self off.

  38. Tehomet Says:
    June 10th, 2008 at 2:56 pm

    So, how many staff do have you have poised to answer support queries on an average day?

Powered by WordPress. Pool theme by Borja Fernandez, modified by DreamHost.
Like WordPress? Consider attending WordCamp LA.
Entries and comments feeds. ^Top^