Scalable Virtual Private Servers from DreamHost are a great way to ensure that you’re never paying more or than you need for your hosting.
You might want more power than shared hosting can provide, but may not really need the resources and cost associated with a dedicated server. With a VPS from DreamHost you can scale your hosting’s available memory allocation in real time and immediately see how those changes affect the bottom line.
And how do you know how much memory you actually need?
Whelp, we provide all VPS customers with memory usage graphs to show exactly what your site’s been up to.
This is what they looked like:
Pretty great, right? We think so. Or we used to, anyway. Turns out they weren’t exactly helpful in the way that you really needed them to be.
We were giving you a single data point to explain your memory usage – what’s known as the resident set size. While that data is useful, the RSS represents just a portion of your overall memory usage – it’s really not enough data to make informed decisions about resource allocation.
Our new graphs look like this:
We’re now showing memory usage trends across your RSS (“Actual Memory”), memory used by your disk cache (“Cached Memory”), and the sum of both RSS and disk cache (“Total Memory”). That Total Memory line is really a much better indicator of where you’ll need to be.
Please note that applications can reclaim memory from cache until their RSS hits the memory limit that you’ve specified – much like any other Linux system.
We capture snapshots of your VPS metrics every 15 minutes. We’re working hard to increase the interval at which we sample data, and some progress on that particular enhancement is already in the works.
We’ve also revamped some code that will adjust and tune Apache and Nginx configurations appropriately based on your specified memory allocation. These modifications should help to improve memory management somewhat to reduce the likelihood of out-of-memory conditions. The end result is that any sudden spikes in memory usage should now occur much less frequently and be handled a bit more gracefully when they do.
We still have more plans for making our VPS reporting even better, but we wanted to get these improvements out the door to make sure our VPS customers had the best data available as quickly as possible.
We hope this helps!
February 21st, 2012 at 2:38 pm
My tab freezes up when I visit this page now. Chrome 18.0.1025.33 browser.
February 21st, 2012 at 2:39 pm
I should add that it looks awesome and I’m looking forward to seeing it. Prior to this, I would get reboot messages from time to time and my memory bar was never anywhere near the limit I had set.
February 21st, 2012 at 4:14 pm
Looks like the Y-axis (memory used) numbers are off by a factor of 5-6x. I assume this is the same VPS yet RSS is showing an average of less than 100MB before and now showing an average of 400MB.
I’m seeing this on my own VPS where the new graphs are showing 5-6x the memory numbers than before. I can SSH in and see the actual numbers via ‘top -c’ and they’re showing that indeed the graph is off by 5-6x.
February 21st, 2012 at 4:29 pm
Graphs are way off here too, you can clearly see the increase on the monthly screen: http://i.imgbox.com/aaso1Z1n.jpg
This isn’t on, my PS keeps going down and the load/traffic hasn’t increased, if anything it has decreased.
Clearly just a scam to get us to pay more. Sorry but I have no money to pay more so unless you revert these changes I’ll have to find another PS provider that doesn’t take down their customers sites because they can’t test changes before unleashing them on their poor customers.
Might aswell be on shared, what a joke.
February 21st, 2012 at 4:59 pm
Same thing happened to me as Darren described….DH, are you trying to blackmail us? to make us pay now more for the same thing???
Unbelievable
February 21st, 2012 at 5:08 pm
For something like these reporting graphs DreamHost needs to inform VPS customers BEFORE implementing the changes. The new reporting scheme took place around Feb 18, UI wasn’t updated until today Feb 21.
We’ve been posting about this in the forums since Feb 18th. http://discussion.dreamhost.com/thread-134266.html
And fix the RAM numbers on the left side of the graph. Pretty please with sugar on top.
February 21st, 2012 at 5:15 pm
I see no pretty graphs at all under Firefox 10.0.2.
The tabular data display, but no graphs.
February 21st, 2012 at 5:21 pm
@Darren @Concerned We’re going to look into why the new graphs are reflecting such dramatic changes for some people.
Also note that nothing at all has changed with how we handle the VPS itself… just the graphs. If your VPS was working fine before, it should still be working fine now. If that’s not the case, let us know.
February 21st, 2012 at 7:58 pm
I am seeing the same graph increases as Darren.
I understand that Total is the sum of Cached and Actual. Is Actual always greater than Cached? It appears that way from the chart but see what ottodv has posted in the thread referenced by baron:
http://discussion.dreamhost.com/thread-134266-post-150165.html#pid150165
February 21st, 2012 at 8:59 pm
New graph is showing me the same weird slope Darren has…. Totally strange… Not sure I like this at all. IMHO this should have been tested better prior to going live.
February 21st, 2012 at 9:02 pm
Noticing the same jumps on my graphs as well and had my site go over my set limits a few times in the past 2 weeks when these changes were implemented even though my sites were not receiving any increases in traffic. What’s going on?
February 21st, 2012 at 11:59 pm
Memory reporting appears to be fixed. Graphs now working correctly. Thanks DreamHost!
February 22nd, 2012 at 2:09 am
Again, like Darren and Omkar and Chris, I have a ridiculous increase in memory usage represented, when there have been no changes to the site. I can’t afford to pay more either!
February 22nd, 2012 at 4:35 am
Same thing happened to me as Darren described and others.
I also remove biggest website on account to other server 5 days ago and memory is still higher then normal :(
Is it possible to back from VPS to normal pack?
February 22nd, 2012 at 4:40 am
The VPS chart looks better for me now (the Actual is in line with what my Total used to be).
Questions:
1) Am I paying for Total or for Actual?
2) The MySQL PS also need their graphs fixed
February 22nd, 2012 at 6:29 am
This is nice. However, I still find that I need to keep my memory resources 140 MB higher than the highest daily spike to keep from getting your emails that I need more resources. I’ve never gotten a satisfactory answer for why this is.
February 22nd, 2012 at 8:06 am
Is there any way for us to manage our VPS cache memory? My cache memory is double my actual memory and there’s no way of reducing/eliminating it without restarting the VPS. I hate paying for things I’m not always using….
February 22nd, 2012 at 8:06 am
I still see no graphs under Firefox 10.0.2 on Mac OS 10.6.8. I have to ssh in and use free -m to find out current memory usage.
The graphs DO display under Safari 5.1.2 (6534.52.7) on Mac OS 10.6.8.
February 22nd, 2012 at 9:01 am
@Josh the interval at which the metrics are collected is still infrequent enough to where we will miss sudden spikes in usage. The memory controller itself is more of a meter and not a historical tracker of time series data so it catches usage as memory is requested over your threshold. If you are getting emails you most certainly are exhausting all “Actual” (or RSS) meaning your guest had already dumped it’s caches to make room for application memory and despite that hit the threshold you set in the panel or via our API.
@Jon You don’t need to worry too much about cached memory, it’s how linux works. If your applications need more physical memory cache will shrink to accomodate the request. Cached memory is important because it keeps recently read files in memory instead of having to do an expensive disk write which would be considerably slower.
February 22nd, 2012 at 9:02 am
@Jon should have been phrased “avoid the need to do an expensive disk *read* which would be considerably slower.
February 22nd, 2012 at 9:12 am
@Shelly Clear you caches on Firefox. I had the same issue with Chrome, with Firefox working just fine. After I cleared cache it was ok in both browsers. I’m talking with Dev to see if we can make that a non-issue.
@Omkar You pay for what you set in the panel, the graphs are simply there to help you see how your machine is using the memory you are already paying for.
February 22nd, 2012 at 9:14 am
I’ve been having the same problem Darren described, for months. Kept increasing memory allocation and paying more and more for it. Still was getting emails about memory limit being exceeded. Eventually I decided not to pay more but started putting my sites with another provider – I went from about 30 sites on DH to just two now – and I’m still getting the ‘memory limit’ messages every other day or so.
Support was no help at all, they expect you to debug what is causing the memory spikes on your own. Actually removing the sites has made almost no difference and the two that remain are tiny, almost no-traffic sites.
To me it seems like there is an incentive for DH to make shared hosting performance really bad so people will then go over to a VPS to see if that helps – when it doesn’t (much) the only thing they can do is increase the memory and pay more and more. It never seems to quite work though…
I am actually getting way better performance with another host on shared hosting than I did with DH’s VPS – so very soon it will be ‘goodbye DH and stick your VPS memory messages where the sun don’t shine’.
February 22nd, 2012 at 9:17 am
I’ve asked DreamHost for clarification of their memory monitoring in the past. There was always a discrepancy between these graphs and the values you can get from the shell.
Please correct me if I’m wrong, but the new Total now appears to be consistent with the ‘used’ memory reported by running ‘free -m’ in a shell. Free also shows ‘cached’ which is consistent with the new Cache graph. The new Actual graph appears to be Total – Cache (as stated). This value is also shown when running ‘free’ (the second row under ‘used’).
Notice that running ‘top’ will show you RES (resident) memory, which is, confusingly, not quite Actual.
I’m all for more information, but this doesn’t seem relevant, since the OS controls cache (right?). Actual is the only metric we actually need to monitor, in order to set our RAM allocation. Please explain how the new cache/total values are useful or important in setting the RAM allocation.
February 22nd, 2012 at 9:18 am
@Omkar I just checked a few accounts with MySQL VPS machines and I see the updated graphs. Try clearing your browser cache and see if that does the trick. If clearing the cache doesn’t take care of the problem please submit a support ticket so we can straighten it out for you!
February 22nd, 2012 at 9:22 am
@neb If you see that your actual usage is dominating your total memory usage it means that the system can not hold in disk cache to provide efficient access to that recently read files that are likely to be accessed again in the near future. With little to no disk cache available to your system you will see more reads from actual disk and that can reduce the speed at which some operations can be preformed.
February 22nd, 2012 at 10:55 am
Thanks Kyle. I am seeing the updated MySQL PS graphs now.
February 22nd, 2012 at 11:07 am
@Kyle – Thanks for the clarification. From what you describe, we’d likely get optimal performance from our VPS if Total is at or below our max RAM allocation. This makes sense, but is a bit misleading. Linux will generally try to use all available (non-resident) memory for disk cache. The result is that, as you add more RAM, the Cache and Total will generally increase as well, even if files are just hanging around in your roomy cache. Still, at least for Web servers, more cache can yield better performance.
Thanks for making these new metrics available.
February 22nd, 2012 at 6:57 pm
Nice charts. I was confused by memory spike a couple of days ago. I actually made some changes and was puzzled by why it affected memory usage so much. Now it is clear that you guys were making updates and displayed total memory as actual for a while. I like the new version much better than old one. But I can see where people might get scared/confused with Total Memory, although in reality they are being charged for Actual Memory only. I think if you can make color changes to make Total Memory look more subtle, or not colored in as Actual memory it should help. And it should be an easy change with Flot that you’re using for this. Great job, glad you’re working on making things better for us!
February 23rd, 2012 at 1:43 am
Regardless of the quibbles on here the graphs are a great enhancement over the previous versions – which were perfectly fine for purpose anyway. Some people always to quibble over nothing. For me Dreamhost offers superb cost to performance ratio and I’m moving more sites to our Dreamhost VPS accounts.
Keep up the good work guys. Keep improving.
February 23rd, 2012 at 2:55 am
I love the changes to the graphs, but it would be really helpful to be able to set a timezone under the actual server time. I think you’re about 8 hours behind me so it’s hard to see exactly what time relates to what at a glance.
Thanks DH!
PS The images I’ve looked at in the comments show high caching levels – reduce the amount of RAM you’re set at and you will reduce caching memory.
February 23rd, 2012 at 9:37 pm
Why do you want to “increase” the interval that the sampling takes place? Don’t you want to decrease it?
How do I control the disk cache?
February 26th, 2012 at 2:30 am
S.o.S .. Just returned home page .. Internal links are all inactive portal … What do I do? All pages are showing 404 errors – Http not found.
February 26th, 2012 at 2:31 am
http://www.estudobiblico.org – http not found – S.o.S .. Just returned home page .. Internal links are all inactive portal … What do I do? All pages are showing 404 errors – Http not found.
February 26th, 2012 at 8:53 am
Is anyone else only seeing the last three days of history being stored and displayed? Even when I switch to month view, all it ever shows me is the last three days.
February 26th, 2012 at 6:17 pm
Same here, before that change, everything running nice… Since that change, 2 reboots for lack of memory… No change in config, no change in traffic…
February 28th, 2012 at 2:53 am
very nice and interesting article!
everyone like it…
thanks for the information…
February 28th, 2012 at 2:15 pm
I’m seeing the same jump is used memory that other users are reporting. In addition, since the change was made I’ve received almost daily “Your VPS needs help, we’ve rebooted it for you.” emails from DreamHost.
Doug
February 28th, 2012 at 5:46 pm
The graphs are more useless than before. You can’t zoom (previous feature) and sampling every 15 minutes doesn’t mean much when the data shown to the VPS customer is hourly (a very rough hour too).
I have “a lot” of the same issues mentioned above and I complain about them “a lot”. Nobody at DH is listening. In recent months I’m using 3 times the memory for the same traffic and processes that haven’t changed enough in the past year to even come close to a need of 3x.
I frequently receive messages that my VPS was restarted for an excess in allocated resources. How much memory do I need? The new graph doesn’t tell me anything the old graph didn’t. It tells me less. The email simply says, “increase” and the wiki info it references has bogus info. I still have a support ticket 2 weeks old about the VPS wiki info and nobody has answered after asking 3 times for reply.
Dreamhost VPS was a great buy when I signed up a few years ago. It’s terrible now and the quality of support has gone downhill fast in recent months.
February 29th, 2012 at 7:57 am
Ps6227 down.
50 hours and my server is dropped.
Please solution the problem with this server. My business depends on it. I am losing a very significant amount of money due to the fall of all webs. Several customers have been angry enough and I lost. It is tremendous damage to my business.
March 5th, 2012 at 1:39 pm
I like the graphs, but could you please change the colors? The plot lines are too difficult to tell apart as the colours are too similar. Thank you.
March 7th, 2012 at 12:13 am
Question: Should I set my allocation based on:
[1] Actual Memory (250MB or less) or
[2] Total Memory (600MB or less)?
March 7th, 2012 at 3:31 pm
In my case, it seems my graph has no data between February 8, 2012 and March 5, 2012. Did anyone notice something similar? Anyway, I already contacted support about it.
Just like Horace above, I want to know if we should set our memory allocation based on Actual memory (in red) or or the others (cached, total).
In other words, please confirm that the memory we are paying for is the “Actual memory” that is in red on the graph. Thanks.
March 8th, 2012 at 6:53 pm
http://www.linuxatemyram.com/
March 12th, 2012 at 7:30 am
Oh, thank goodness for this! Not sure if it will catch my memory spikes, but it’s nice to see these improvements!
March 12th, 2012 at 1:03 pm
From the looks of things, these new graphs are about as worthless as the old graphs.
By looking at my graphs, the “total memory” has shown a steady rate of 500mb all day, every day. My “cached memory” is around 300mb and my “actual memory” is just under 200mb. Sounds pretty straight forward, doesn’t it?
If I look in the tabular version, the graph information is verified. “Cached” is 317, “Actual” is 164, and “total” is 481. CPU Usage is 0.02. With slight fluctuations, it has read like this for weeks.
YET…
If I attempt to resize my resources to 800mb, I get the notice that my resize has “FAILED”, and that my “actual usage” is 928MB If I try for say, 1000mb it fails again telling me my actual usage is now 1241mb. If I run the damn thing all the way up to 4000mb, THEN it seems to have no problems.
If I leave it at 4000mb for an hour, my graph STILL says I am using around 500mb of total memory and so dies the tabulated version.
March 17th, 2012 at 2:29 am
week and month always report the same data – but graphs are much better – thanks!
March 19th, 2012 at 8:38 pm
Warning: VPS’s apparently do not get backed up. Something went wrong with my hosting (DH is being tight lipped about it), and backup restores are not working at all.
March 19th, 2012 at 10:13 pm
Just chiming in to say that yep, same problems here. Been having them for *months*, graphs or no graphs, and support seems to think that sending me the same KB article over and over will somehow yield new information. (I’ve done everything the KB said to. Still got the over-memory errors/emails.)
I get jack for traffic. There’s absolutely no reason why I should be getting errors and overages, but I do. Spent most of tonight, in fact, deleting old databases and backing things up in prep to move elsewhere, which is sad since I’ve been here since the beginning. (Almost a decade and a half. Brought in *tons* of referrals over the years, right up until it started to suck.)
March 21st, 2012 at 11:50 am
I’m with Elizabeth. We are moving all of our websites to a new host as soon as possible. Dreamhost Support is not helpful whatsoever and has offered no aid other than sending us the same wiki articles.
March 21st, 2012 at 3:58 pm
After some consideration, I have decided that I, too, might make the move to another host.
It is obvious that there is a serious disparity between what resources are actually being used and what the graphs say we are using. It is next to impossible to economically determine where we need to be at any given moment. The more I raise my resources, the more it says I need, yet the graph says I am not needing that much.
I’ve given up. I can get a VPS account with more guaranteed resources at SPRY.COM for half the price. I would have full cpanel access, Shell access, FTP and if I really had a problem I could just pick up the phone and talk to someone right there on the spot.
March 23rd, 2012 at 3:48 pm
Glad to hear that I’m not the only one on the back-end of all of this mess. I’ve been battling with DH for almost two years about my memory resources and why I would constantly get out-of-memory errors and nice emails saying “We had to force your server to reboot”. The only reason that happens on my site that maybe gets 100 hits a day max is the disk cache which I have no control over and DH is not willing to admit is their problem. I also will be joining the hunt for a different hosting provider.
March 24th, 2012 at 12:49 am
We do not mind this useless reporting.
We would prefer that you manage to keep everything on without outage as it happens every month now…
March 24th, 2012 at 12:52 am
Hey guys, why don’t you use PSMANAGER ?
http://www.7is7.com/software/dreamhost/psmanager.html
Because Dreamhost doesn’t advertise to anyone that it exists ? Yes, great idea to pay more than what you need…