As some of you already surmised from this old post, what I was really talking about was pretty simple:
Should we get our own data center?
I was planning on getting around to that eventually, but the post was getting pretty long, and boring, so I thought I’d cut it off abruptly and break it into a two-parter. That also gave me the added advantage of being able to cleverly reverse the post title!
So, here’s the thing.. when deciding whether to build or buy forget all that stuff I said back then. That was two weeks ago, man… ancient history! This is the blogosphere, woman!
Build vs. Buy is not about focusing on your core strengths at all! It’s really just the simple question: “If I build it myself, will it be cheaper?”
But, what is meaning, cheaper?
Cheaper, is meaning, what is your return on investment (that’s ROI for those MBAs in the H.) if you decide to build something instead of buy it?
Sometimes, you should build something even if it costs more than buying it, because maybe by doing it yourself you’re going to do it better, and doing it better results in either lower costs or higher revenues… eventually. It’s all just ROI.
It seems pretty clear that building our own data center would eventually save us money, but the question is when? And more importantly to you, the question is also when will it save YOU money?
The answer to that is of course never. All that saved money is going to escorts and liposuction… FOR ME US!
But anyway, um, who gets the money is irrelevant, right? I think we can all agree that money intrinsically needs to be saved!
Mooooving right along. When you need just one of something, you should definitely buy it if you can. There’s no way you’re going to be able to do it cheaper than the product already on the market in your very first try.
As any fool can plainly see, the solution to the “build vs. buy question” has everything to do with quantity. The more you need, the more you need to build.
“Thank you, Captain Obvious.”
Well, yeah. I dunno, there hasn’t been a blog post in like two weeks, okay? And I haven’t done a lot of thinking recently.
But, even though it’s all just about quantity, “build vs. buy” is still an interesting topic! Because it’s really about how much quantity.
Google didn’t build their own data center when they only had indexed two sites.
Walmart didn’t own their own trucks when they only had two stores.
McDonald’s didn’t own their own potato farms when they only had two served.
The more of something you need, the better the ROI of building it yourself gets!
For example, even though you pay much less for floor space when you build out your own data center, you also generally have to get a lot (several thousand square feet) of floor space upfront, whereas when you colo you can buy floorspace in chunks effectively as small as one square foot!
Buying (or renting) your own data center may be five times cheaper per square foot, but that doesn’t do you a whole lot of good when you’re forced to buy 100 times more than you actually need!
And it’s the same sort of math with everything else you need to build out a data center. When you’re small, you probably don’t need the full capacity of a UPS module, a generator, or a cooling system. Those things are all very expensive, and unfortunately not sold in fractional pieces! You’re also going to need some expertise in nitty-gritty data center stuff.
Stuff like city grid electricity. Air Conditioning. Air flow. Network conduits. Real estate. Installing racks, cages, and ladders and making them seismically safe. Fire supression. Physical security. And uh, I think that’s about it?
People who are good at all that stuff are also very expensive. Fortunately, they are sort of sold in fractional pieces, as consultants… but really it’d be best to have at least one on your full-time staff.
As you get bigger, it’s worth it to do more.
That’s why large large large companies like McDonald’s are “vertically integrated” to the max.. they don’t just own the potato farms; they own the slaughterhouses, the land their restaurants are on, the paper mills that make their napkins, and I assume salt factories and ketchup mines too. They build it ALL.
Starting out though, companies are “horizontally integrated”.. anything they can buy, they do. At places like eMachines, or tiny webhosts, or Westinghouse TVs.. everything but the brand itself is outsourced. When you’re new and small, you’ve got to be nimble and you’ve got to be thrifty. Without the volume, you can’t afford all the capital expense and time investment it takes to make every last tiny sub-component of your business profitable. You especially can’t afford the time.
But in time, things change.
I’d even say that change is one of the most interesting things about growing a business: the slow, inevitable, and continual move “up the food chain”. It’s also one of the most difficult things about growing a business. Because in order to compete, you’re forced to constantly delve into new fields you never considered critical, interesting, or even knew existed back when you thought New Dream Network was just going to make web pages for campus security for the next ten years.
And each time you take another step up that “vertical integration” ladder, it’s a risky proposition. You’re getting into an entirely new line of business. A business with unforseen expenses, risks, and rewards. A business you probably know nothing about but your vendors (now known as “your competitors”) have been doing for decades. And despite all the research, research, research you do, you’ll never really understand what you’ve gotten into until it’s waaaay too late to back out.
It’s scary stuff!
But please don’t be discouraged! One thing I’ve learned so far… if your vendors can do it, you can too. After all, vendors are all idiots, right?
I mean, just look at your web hosting vendor.
April 13th, 2007 at 6:04 pm
Soooo, the verdict is… build or buy? :D
April 13th, 2007 at 6:06 pm
Which option will increase your reliability the most? Pick that one, please.
April 13th, 2007 at 7:09 pm
You folks cannot even seem to manage your current situation – reminder, the old attage ‘people are promoted to their highest level of incompetence’ only holds true for ‘people’ not ‘businesses’ :p
April 13th, 2007 at 9:46 pm
Wow, I’m impressed that you wrote a whole post on scaling and didn’t even use the word once. Better than 99% of the other Web 2.0 junkies these days.
Congrats :)
April 13th, 2007 at 9:50 pm
@dave and @mv hate the game, not the player.
2010 called and wants to tell you that websites are still unreliable. if you’re so pissed why not find a new host rather than flame-baiting?
@josh – save that money and get those hookers and liposuction!
April 14th, 2007 at 1:11 am
Man, if you guys really are planning on building your own data center and want a manager that can ride herd on pretty much everyone he needs to get the best price and quality, you let me know. ;)
April 14th, 2007 at 3:14 am
If renting/buying leaves you and your customers at the mercy or your vendors which might or might not have a conflict of interest and if you guys foresee subscriber GROWTH building a data center is a no-brainer. The more you grow while renting, the more problems you’ll have down the road should should you wish to part ways and build your own.
@greg, if only moving domains, files, and DBs was as easy as you suggest. Even if you are a neat freak, and keep everything organized and all your files accounted for, it still takes the better part of a week to move hosts and get back up and running — not an option to many of us. Not to mention, many of us already paid a full year (or more) in advance.
@Josh, hookers and lipo? Not on my dime. Use the pennies of people who are smart enough not to read this blog.
April 14th, 2007 at 6:21 am
I got a look at the insides of AOL-West in Mountain View CA on the Netscape campus just before AOL took occupancy. Only a single rack of stuff on a floor about 50×75 meters. Power and cooling took about half the building, including a battery farm that went on for days, plus two huge, diesel generators. There were two OC192 fiber connections.
I talked briefly with the planning and operations chief while on the tour. He described issues related to planning, designing, and running such a datacenter that I hadn’t imagined. If you go this route, please be careful. — There be dragons here! … but, you know that.
April 14th, 2007 at 6:45 am
Sorry Greg, but those guys are right. Dreamhost has become decreasingly reliable lately, to the point where I have to pull anything that’s not a personal site. Spend a little time catching up with the growth.
April 14th, 2007 at 7:22 am
Sounds good to move into a new place. I’ve done it a few times over the years. A few things to make sure get done. First you need a great bar and karaoke stage. Nothing boosts employee moral like getting them drunk and then make fun of them. Also if your building your own palace you would need a cornor office with private bathroom (solid gold fixtures optional). And make sure to get permits for the elevator before trying to move in the systems or you will be using a forklift and an open window on the second floor. I almost had to do this in Aliso Viejo. An envelope full of money can speed up the safety inspection process.
April 14th, 2007 at 4:26 pm
This decision about the future of our data center accommodations that Josh discussed in this post is intrinsically linked with every other part of our service. It affects everything we do and has a large part in dictating the future choices we will have.
Suggesting we not make the decision about our next step and instead focus on something else (such as ‘catching up’) just doesn’t make any sense. It’s not possible to split the two ideas apart.
April 14th, 2007 at 10:08 pm
Citation: “and making them seismically safe”
You really believe that that can be done? Interesting as the real big bang will inevitably come. You must have a solid believe in praying to the Gods.
If ever setting up your own datacenter, please move it out of the danger zone where you are now located. Preferable to Canada or Europe. Not Asia too much war there.
Ah well, when the axis of the earth really moves big time we’re all done for.
April 15th, 2007 at 3:14 am
Wish Google would just buy you. Then you can have all the data centers you would ever need.
April 15th, 2007 at 9:17 am
Build it where most of your employees are located. We found a great spot in Brea, but the prez of the company said nope he wanted it close to his house. We lost a lot of great employees that didn’t want to drive that far.
And to the earthquake guy above. There are lots of things to help. The concrete foundation can be made to move with the building. The walls can be designed to fall out and not in. The racks can be bolted into the floor. Etc… I think it is a great idea and hope you are successful.
April 15th, 2007 at 1:51 pm
I think that it great that you are to the point that having this as a problem. But on the other hand, the reliability your services has decreased. So my question is “Do I stick around as a customer because I believe in dreamhost and its principles or do I go to a hosting company that isn’t going to distract me from my core tasks because my site is up?”
Do what you have to properly grow the company but make it happen sooner than later…
April 15th, 2007 at 2:39 pm
@Evocatus: Dude you have problems…go get some therapy or something.
and about the whole Build or Buy thing. Whatever helps the reliablity is what I want because I have had a lot of issues lately and want it to be resolved. I have the smallest plan available and only just less then 15% of that and I’m being told I’m saturating the server….I think they’re overloading on each server. So if building a data center helps that then great.
April 15th, 2007 at 9:02 pm
@Dallas
The huge mistake that businesses like yours continue to make is purchasing/building these centers in ridiculously high real estate areas and in areas prone to blackouts and earthquakes. Staying out of NY, the NorthEast is a far better deal. Further, if I were the capital group on the other side of this deal, I would have real concerns about Google entering this market.
April 16th, 2007 at 4:56 am
I’m agree with Dave & mv. Please do something to make things reliable. I don’t think even 30% of total DH clients use more than 20% of available bandwidth and disk spaces.
Reliability is much more important than having 100+ GB of useless web space that’ll never be used by a lot of DH clients.
April 16th, 2007 at 4:58 am
There are earthquakes, tornadoes, blizzards, and hurricanes all over the country. This is why every major company is not located in Boise. If Josh and Dallas like Cali… Then they should stay in Cali. I do agree with you on the high real estate areas, but if you build a datacenter in 29 palms, good luck getting qualified employees.
April 16th, 2007 at 7:44 am
Do whatever that will make things more reliable.
April 16th, 2007 at 8:14 am
DreamHost – is a core competency of yours building and managing a data center?
I ask because it’s one thing to co-locate. It entirely another things to own and run a data center where you are directly responsible for implementing backup generators, lighting, cooling, wiring etc..
April 16th, 2007 at 11:14 am
From the post “Anatomy of a(n ongoing) Disaster..”:
> Why don’t we have our own data center?
> Because, believe it or not, we’re still not big enough for
> it to make sense. Even now, we only use about 1000 sq ft
> of data center space.. for it to really start to make sense
> to get our own space, we’d have to be using around 2500 sqft
> Mainly because when you buy a data center, you want to get
> one big enough to handle a lot of growth.. and although it’s
> cheaper per square foot than co-locating, you have to pay
> for all the space you’re not using yet.
> And really, The Garland Building is supposed to be an
> excellent place for data centers.
so, would you build a better datacenter, with less power failures, that would let you grow more than twice your size today, and would make you more competitive?
April 16th, 2007 at 9:23 pm
Umm… I think your logic is a bit flawed, and more than a bit difficult to follow.
You must also consider that the purchase price is not the final/actual price of buying most things. There are added costs that can not always be put down as expenses for tax purposes and what not.
Consider depreciation. How quickly over the next 10 years will what you buy depreciate as opposed to what you build.
Consider more than the appraised value of the purchased center because that’s just a “best guess,” although fairly accurate in many cases, is it really accurate enough to make such a serious decision?
April 17th, 2007 at 5:46 am
Well a move could either make or break this business.
If you move, it’s gonna be a hard challenge to move and make it seamless to the point where there is very little downtime. If downtime happens (and with a move I reckon it would be around a day) This would give you even more bad rep and you would lose a HELL of a lot of people and people not joining.
IF on the other hand, you found a way to move with very little downtime, you should have a roaring success. People will stay with you cos they know you’re expanding, and more people would sign up because by that point you should have a better system in place with regards to redundancy and other things.
Personally I think you should move, otherwise you’re limiting you’re ability to grow as a competitor. You have a great business ethic which people like, you just have to get the move spot on and the hosting world is your oyster.
This needs very careful planning but you guys can do it! :)
Good luck!
April 17th, 2007 at 1:52 pm
Build ftw!
April 18th, 2007 at 11:50 am
One added cost of building versus buying is the cost of running the first center during the build process; another (if you are borrowing $$ to build) is the cost of interest on borrowed funds during the build process.
Either way, I’m thinking of transferring my site here, so please be quick so that my site will go real quick (and all my referrals will have plenty of space to move into).
April 19th, 2007 at 1:12 am
Which option will increase your reliability the most? Pick that one, please
April 19th, 2007 at 7:58 am
It scares me to think of the current dreamhosters managing their own data center. I love the value I’m getting right now and they’ve created a great set of features for the money. But to me, with the current team, mo’ growth just means mo’ problems.
All the network, file storage, and other recent issues show that this is not a “high-performing IT organization.” Although the root causes of all the recent issues seem to be discovered and fixed eventually, the fact that it often takes so long to figure out what is wrong and that issues often reoccur after the root cause is supposedly fixed show that they really have limited expertise in what they’re doing. High-performing IT operations have low mean-time-to-repair and a high first-fix rate.
It takes more than some contractors and some recent college grads/dropouts to run a highly effective IT operation; you need people who know what they’re doing running the show making wise architecture and management decisions. At some point in its growth path (hopefully before the decision is made to buy or build), dreamhost will need to begin bringing in-house an experienced management team. Otherwise, we can continue to count on the scope of dreamhost’s issues to expand with their growth.
April 20th, 2007 at 6:44 am
Oh my God! I just finally sever the cord from 1&1.com after their datacenter move takes down all 300+ of my domains registered with them and I see a picture of their “alleged” data center on this blog!
http://order.1and1.com/xml/order/DataCenter
Josh. Do NOT buy this data center! They say it is in the American Midwest but everyone who “allegedly” works there has a British/SE Asian accent.
I’m sure there are other, less frightening images of data centers that could be displayed on your blog.
April 20th, 2007 at 8:31 am
Would never want you owned by GOOGLE… i can see it looking bad… they would proably filter my site and make me put google ads all over it and they would up the prices…….. just stay who you guys are…….
April 20th, 2007 at 10:44 am
“After all, vendors are all idiots, right?
I mean, just look at your web hosting vendor.”
That last bit made me laugh. (Oddly appropriate considering my alias…)
I’m glad you folks have managed to keep you sense of humor.
April 22nd, 2007 at 7:26 pm
These days colocation providers have refocused their efforts on regaining profitibility not only for this point forward, but also for years past. This is going to change the buy vs build model for anyone with more then a small cage and doesn’t have to worry about Rock-Solid SLAs.
My costs for colo power have gone up 10 fold, and each cabinet went up 40% on this year’s contract renewal. Burstable Bandwidth did on the other hand go down from $200 per mb/s to $160.
So, for my itty bitty little ASP service currently sitting in 3 cabinets, but has a 99.9% SLA to meet – I need to stay with my colo… But my staging systems (without SLA) are comming home soon…
I’d rather save the money so I can invest it back into my engineers who make my clients happy.
DreamHost – You’re not 100% reliable, but you also don’t change 100% reliable prices. You go way above the expected value proposition, and I’m happy that you host all my personal sites… And thats from a guy who runs web servers (not 1300, but enough) as their full time job!
May 15th, 2007 at 11:05 am
Article: build or buy (do it all yourself or outsource) t-shirts, printing, etc…
I ran across this blog post recently from DreamHost.
DreamHost Blog ……
September 11th, 2008 at 1:42 pm
[...] Here at DreamHost, we’re as serious about web hosting as we are about ping pong. And if we need more room for servers — or a quick doubles game — we’re totally in favor of expansion. [...]
September 13th, 2008 at 8:01 pm
[...] Here at DreamHost, we’re as serious about web hosting as we are about ping pong. And if we need more room for servers — or a quick doubles game — we’re totally in favor of expansion. [...]