100% memory usage, no trafficked sites.

Z

Zygar

New Member
Greetings;

I've been having an issue with a VPS that I recently bought, on the VPS-M plan. I budgeted for 128mb of RAM, seeing that there was a knowledge base article which implied that some clients hosted dozens upon dozens of low traffic, basic html domains on this plan with no issue - "We have customers on VPS-XXL plan with single web site in their VPS (i.e. large forums) and customers on VPS-M with couple hundreds mostly static and low traffic domains."

My issue is as such: Yesterday morning, my server memory usage was 75/128mb. Yesterday evening, it was 100/128mb. I disabled spamassassin (doing this caused my server to skyrocket to 100% memory usage for quite a while!) , etc, and restarted the apache process, and it went down to about 90/128mb. This morning, I logged on and lo and behold, it was sitting at 128/128mb! Another restart of httpd later, and it's down to 90/128mb. I followed this by a graceful reboot of the server, and it went down to a staggeringly small (comparitively) 68/128mb! (EDIT: 72/128 now!)

Here is a log of my memory at 90/128mb:
http://zygar.co.nz/memory.txt

and here it is at 72/128:
http://zygar.co.nz/memory2.txt

Fact of the matter is, troubleshooting this is way beyond my skill level right now. I am pretty good with computers in general, but Linux and web servers is something I'm new to, (I mostly dabble in Windows and hardware) so I feel somewhat like I've stepped into the deep end.

What seems to be happening is that server memory usage starts off low, and begins to increase, without ever decreasing. It's like some processes are taking my memory and not letting it go. Considering that I have one Drupal site which is currently disabled, and the rest of my site consists of mostly a dump of some small files right now, this is baffling me. I have barely any traffic, yet my memory usage climbs, and climbs, and climbs until I have to restart the server. Any help on this matter would be very much appreciated.

I'm a quick learner, and as I mentioned, not a computer newbie so don't hesitate to throw big words at me as long as they aren't too Server/Linux specific. I am extremely new to dealing with servers, so _any_ helpful feedback in that regard would be highly appreciated as well.

(Also, it should be mentioned that yes, I am using free -m to find out my memory usage. As I said, quick learner! ;) )

EDIT: Again, it climbed. 79mb now, and the log is here: http://zygar.co.nz/memory3.txt
I also realized how much I sound like one of those ancient folks who calls tech support and goes "I've been using computers since before you were born, young man, and I am telling you that my jiggawatts are broken." I can assure you this was not my intention, just trying to clarify that I at least have a rudimentary idea of what I'm doing when I use a computer, and I'm not adverse to command lines or anything; and that I am capable of picking up on anything computer-related pretty quickly even though this is in no way my field of expertise.

EDIT 2: And from what I can tell, for some reason apache seems to be multiplying itself and then hogging all of the ram to itself?
edit 3: what on earth, it's shot up to 88mb already.
http://zygar.co.nz/memory4.txt
 
Hello Zygar and welcome to Knownhost :)

After taking a look at one of your text files there I see you are running Cpanel 11. Personally I do not even understand why they sell the M version of the VPS with Cpanel as it will use 128MB of ram all by itself.

I disable cpdavd and cphulkd on my VPS as I don't have any users that use cpdavd (web drive) and cphulkd just sits there taking up memory and doesn't do anything for me.

To stop cpdavd: /usr/local/cpanel/etc/init/stopcpdavd
To stop cphulkd: /usr/local/cpanel/etc/init/stopcphulkd

This will stop them until the next reboot or the next Cpanel update.

To stop them from starting with the next reboot (this will have to be done after each Cpanel update) then edit the file /etc/init.d/cpanel and fine the lines where they start and comment them out. Should like like this:
# echo -n "Starting cPanel brute force detector services: "
# daemon /usr/local/cpanel/etc/init/startcphulkd
# echo
# echo -n "Starting cPanel dav services: "
# daemon /usr/local/cpanel/etc/init/startcpdavd
# echo

Now Apache seems to be using 7% of your ram for each child instance which is very high. Mine is only using 1.5% out of 256MB which should still only be 3% on a 128MB system. Are you running modsecurity or something? I found that when I ran modsecurity with the Gotroot rules Apache used a HUGE amount of ram. If that is the case with you then I suggest changing to modsecurity's core rules instead.

These will help some but, as previously stated, you will probably have to upgrade your ram in order to be able to run Cpanel happily.

Hope that helps
 
Hello!

In general we don't recommend the VPS M w/ cPanel/WHM but the reason the VPS M exists is because we sell addon's like RAM and Disk Space a la carte which in turn does make sense. Many people don't require the amount of Disk Space or RAM in the base plan so this allows more flexibility.

I too would recommend upgrading the memory or the base plan to the VPS L. Happy Holidays!

Joel
 
Thank you both for the quick replies! I'll look at upgrading the RAM in the future, but in the interim I have disabled those two extraneous cpanel processes and it dropped my memory usage significantly - I still can't figure out why my memory usage just climbs, and climbs, and climbs even while not doing anything, however. It does seem to be some setting or module with apache, but unfortunately apache is one of the areas that I am new to - how do I change the settings on this mod_security you speak of, or make apache less bloated? The apache processes were climbing up to 12% each at one point before I restarted the server. This seems excessively high and I can't figure out why, as I'm running apache pretty much stock standard and there's very little traffic to speak of.

Basically, it looks like Apache is running away and taking all my RAM with it - and I'd wager that if I upgraded my RAM, it would chew up all of that as well.
 
You can limit the number of child processes Apache spawns, but the fact of the matter is you need more RAM.
 
Dan said:
I disable cpdavd and cphulkd on my VPS as I don't have any users that use cpdavd (web drive) and cphulkd just sits there taking up memory and doesn't do anything for me.

To stop cpdavd: /usr/local/cpanel/etc/init/stopcpdavd
To stop cphulkd: /usr/local/cpanel/etc/init/stopcphulkd

This will stop them until the next reboot or the next Cpanel update.
Click to expand...

cphulkd can be enabled / disabled via the security centre of WHM. I'd presume that this would stop the config being over written when Cpanel is updated and take away the need to manually update the cpanel file after update.

Funnily enough, my cphulkd was already disabled, I wonder if the Configserver firewall I'm running (with LFD) disabled it when I installed CSF?
 
khiltd said:
You can limit the number of child processes Apache spawns, but the fact of the matter is you need more RAM.
Click to expand...

I'm not entirely ruling out more RAM, but on a site with such little traffic, why would each Apache instance take up 12% of my memory? There has to be some way to slim it down, modules that I don't need or something, to make the most of my memory.
 
I may suggest you Lighttpd , very happy with it, you can see my server running yet in the Nagios demo post I have made in Howto section, remove spamassasin and if you're on windows (correction that's also available to linux) I suggest you Spambayes the most better bayesian adaptive filter I have tried ever, you train it and that's a real shark then and its runs on clients side not on the server so you save lots of resources, dont runs avast or any other useless linux stuff like that, so if you're using Qmail, uninstall it and try postfix, use so far less procs and mem than qmail
 
Zygar said:
I'm not entirely ruling out more RAM, but on a site with such little traffic, why would each Apache instance take up 12% of my memory? There has to be some way to slim it down, modules that I don't need or something, to make the most of my memory.
Click to expand...

Have you recompiled apache only using the modules you want? I recently had to add some modules and ram usage went right up, so I suppose the opposite would be true - get rid of those you don't want and usage may go down.
 
pmcwebs said:
Have you recompiled apache only using the modules you want? I recently had to add some modules and ram usage went right up, so I suppose the opposite would be true - get rid of those you don't want and usage may go down.
Click to expand...

I'll go look for instructions somewhere on how to do that, thanks. Hopefully I can trim it down to about 3-5% usage per process, which would be ideal.
 
Ah, I am getting the impression that it seems to be something with the PHP module in Apache - every time I load the Drupal site that I'm working on, one of the Apache processes goes from 6% to 13% memory usage - and doesn't let that memory go. Each subsequent loading of a PHP page causes another apache child process to bloat up. Is there a way to tell Apache to release that memory after using it? (At this point, it's very likely that I'll upgrade my RAM as well but I want to deal with this first.)

EDIT: Yowch, each load of a PHP page increases memory usage by 20mb. This is insane.
 
This seemingly unnecessary redundancy is done in the interest of performance. If you had a single Apache process handling every request then every visitor would have to sit around waiting for that process to handle however many requests were queued up before they got there, and that would get annoying very quickly. Whether you have any traffic or not, this is normal behavior for any web server. Extraneous processes will typically be reaped after a set amount of idle time.

128MB is not enough for cPanel and a standard compliment of services, period. Add a gigantic, sprawling, unoptimized application framework like Drupal on top of that and you don't even have fumes to coast on. You can prune modules all you like to eek out every bit of space you can, but you're not going to escape the simple fact that you don't have enough RAM. Maybe if you ditch the control panel and just run Apache by itself, but even then, 128 is pretty much nothing.
 
In that case, it looks like I'll probably be upgrading in the new year, if I trim everything down (I'm sure there are one or two modules that I don't need) should 256mb be sufficient, bearing in mind that I am not going to have a great deal of traffic?
 
I have also noticed that when the VPS-M is restarted, the initial memory usage is approx 76MB out of 128MB. About 15 mins later, it moves up to approx 88MB or so. There is no active website or much other activity on the VPS during this time.

When I checked using "ps auwwwfx", initially there were 7 of these processes listed.

/usr/local/apache/bin/httpd -k start -DSSL

Later there were 8 listed, hence the increase in memory usage.

The 1st uses 9.6% (approx 12MB). The 2nd uses 5.3% (approx 7MB). The other 6 each use 8.7% (approx 11MB).

I would like to know more about this process.

How do they work? Does each one handle a new web connection?

Can each process handle child processes? If so, how many?

Do we need 8 of these processes running at one time?

Why is there a difference in memory usage for the 1st, 2nd and rest?

In an earlier post, Dan said that on his 256MB VPS this process uses 1.5% (approx 4MB). On the 128MB VPS, it is using approx 7 - 12MB. Why the difference?

Dan, are you using Apache 1.x or 2.x? How about yours, Zygar?

The above is on Apache 2.x. Support recompiled it to cater for PHP 5.2.5 and MySQL 5.0.27 and other default settings. SpamAssassin is disabled. No other special applications, just the default configuration.
 
Zygar said:
In that case, it looks like I'll probably be upgrading in the new year, if I trim everything down (I'm sure there are one or two modules that I don't need) should 256mb be sufficient, bearing in mind that I am not going to have a great deal of traffic?
Click to expand...


Zygar it sounds like you are headed in the right direction to me. Why your system is using so much more ram per Apache process I have no idea but do as Khiltd suggested and recompile Apache/Php and install just the modules that you need. And yes 256MB should be sufficient.

mylinear said:
I have also noticed that when the VPS-M is restarted, the initial memory usage is approx 76MB out of 128MB. About 15 mins later, it moves up to approx 88MB or so. There is no active website or much other activity on the VPS during this time.

When I checked using "ps auwwwfx", initially there were 7 of these processes listed.

/usr/local/apache/bin/httpd -k start -DSSL

Later there were 8 listed, hence the increase in memory usage.

The 1st uses 9.6% (approx 12MB). The 2nd uses 5.3% (approx 7MB). The other 6 each use 8.7% (approx 11MB).

I would like to know more about this process.

How do they work? Does each one handle a new web connection?

Can each process handle child processes? If so, how many?

Do we need 8 of these processes running at one time?

Why is there a difference in memory usage for the 1st, 2nd and rest?

In an earlier post, Dan said that on his 256MB VPS this process uses 1.5% (approx 4MB). On the 128MB VPS, it is using approx 7 - 12MB. Why the difference?

Dan, are you using Apache 1.x or 2.x? How about yours, Zygar?

The above is on Apache 2.x. Support recompiled it to cater for PHP 5.2.5 and MySQL 5.0.27 and other default settings. SpamAssassin is disabled. No other special applications, just the default configuration.
Click to expand...

Mylinear I can't answer about the processes of Apache perhaps someone else or Google can.

Don't know why there is such a large difference of memory usage between the two systems either although there are dozens of different modules that you can can install and most likely it's one or more of those causing the difference. And yes I am running Apache 2.x as well.

Am headed over the pass here pretty quick for Christmas but we can revisit this more if you'd like after I return.

Merry Christmas everyone!
 
mylinear said:
Does each one handle a new web connection?
Click to expand...

Yes. And since it would be pointless to waste time quitting and restarting them all, they stick around to handle future connections.

Can each process handle child processes? If so, how many?
Click to expand...

Children are not spawned recursively if that's what you're asking. The number of children allowed is specified in httpd.conf

Do we need 8 of these processes running at one time?
Click to expand...

8 isn't very many. Lower the MaxChildren directive to one or two and see how much you enjoy the performance you get as a result.

Why is there a difference in memory usage for the 1st, 2nd and rest?
Click to expand...

A plain HTML page will require significantly fewer resources to read in and spit out than a PHP script which requires parsing, allocates additional resources, opens database connections, invokes endless strings of mod_rewrite rules, etc. The latter all being exceptionally common occurrences amongst web application frameworks that strive to be all things to all people. A frightening number of PHP developers have never worked in any other, less forgiving, language and as a result their code is commonly completely unoptimized and prone to memory leaks.
 
Just wanted to thank you for your detailed response, khiltd. It gave me quite an insight into how Apache works; and hopefully I'll be able to get to optimizing it for the sites that I'll be running.

Just want to clarify one thing - Apache seems to load the modules used for parsing PHP only once, the act of which bumps the process up to a maximum of 12-14% memory usage. I assume that the reason it sticks at that memory usage is not because it's actually doing anything, and just because it's holding the module in memory in case it's needed again. Does this mean that hosting say, two or three different, low traffic Drupal sites on one VPS would use roughly the same amount of memory as a single site? Due to the processes getting shared and reused? (MySQL memory usage aside.)
 
If you're running PHP as an Apache module then I don't believe there's any reason for its shared libraries to be loaded more than once as current versions should be fairly re-entrancy friendly, but I won't swear to that. If you're running it as a CGI application then that can eat up more of everything, sacrificing speed for security. Then there's all the wacky fcgi options which are tremendous resource hogs. It really depends on how your installation was built and configured.
 
khiltd said:
If you're running PHP as an Apache module then I don't believe there's any reason for its shared libraries to be loaded more than once as current versions should be fairly re-entrancy friendly, but I won't swear to that. If you're running it as a CGI application then that can eat up more of everything, sacrificing speed for security. Then there's all the wacky fcgi options which are tremendous resource hogs. It really depends on how your installation was built and configured.
Click to expand...

I don't think I have it running as a CGI application, my installation is pretty much default. So having multiple PHP sites won't have significantly more impact (in terms of apache process size) than running one site? That's good to know, thanks. :)
 
You must log in or register to reply here.
Top