Where's my memory?! Nginx + PHP-FPM front end webserver slows to a crawl...
- by incredimike
I'm not sure if I have a problem with a memory leak (as my hosting company suggests), or if we both need to read http://linuxatemyram.com. Maybe you clever people can help us out?
This is a front-end webserver VM running essentially only nginx & php-fpm on RHEL 5.5. This server is powering Magento, a PHP eCommerce thinggy. The server is running in a shared environment, but we're changing that soon.
Anyway.. after a reboot the server runs just fine, but within a day it will grind itself into nothingness. Pages will take literally 2 minutes to load, CPU spikes like crazy, etc.. The console is even sluggish when I SSH in. It's like my whole server is being brought to its knees.
I've also been monitoring the DB server via top and tcpdumping incoming traffic. The DB stays idle for a good portion of that "slow" load time. When i start seeing queries coming from the front-end server, the page loads soon afterward.
Here are some stats after me logging in during a slow-down, after restarting php-fpm:
[mike@front01 ~]$ free -m
total used free shared buffers cached
Mem: 5963 5217 745 0 192 314
-/+ buffers/cache: 4711 1252
Swap: 4047 4 4042
[mike@front01 ~]$ top
top - 11:38:55 up 2 days, 1:01, 3 users, load average: 0.06, 0.17, 0.21
Tasks: 131 total, 1 running, 130 sleeping, 0 stopped, 0 zombie
Cpu0 : 0.0%us, 0.3%sy, 0.0%ni, 99.3%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 0.3%us, 0.0%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 6106800k total, 5361288k used, 745512k free, 199960k buffers
Swap: 4144728k total, 4976k used, 4139752k free, 328480k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
31806 apache 15 0 601m 120m 37m S 0.0 2.0 0:22.23 php-fpm
31805 apache 15 0 549m 66m 31m S 0.0 1.1 0:14.54 php-fpm
31809 apache 16 0 547m 65m 32m S 0.0 1.1 0:12.84 php-fpm
32285 apache 15 0 546m 63m 33m S 0.0 1.1 0:09.22 php-fpm
32373 apache 15 0 546m 62m 32m S 0.0 1.1 0:09.66 php-fpm
31808 apache 16 0 543m 60m 35m S 0.0 1.0 0:18.93 php-fpm
31807 apache 16 0 533m 49m 30m S 0.0 0.8 0:08.93 php-fpm
32092 apache 15 0 535m 48m 27m S 0.0 0.8 0:06.67 php-fpm
4392 root 18 0 194m 10m 7184 S 0.0 0.2 0:06.96 cvd
4064 root 15 0 154m 8304 4220 S 0.0 0.1 3:55.57 snmpd
4394 root 15 0 119m 5660 2944 S 0.0 0.1 0:02.84 EvMgrC
31804 root 15 0 519m 5180 932 S 0.0 0.1 0:00.46 php-fpm
4138 ntp 15 0 23396 5032 3904 S 0.0 0.1 0:02.38 ntpd
643 nginx 15 0 95276 4408 1524 S 0.0 0.1 0:01.15 nginx
5131 root 16 0 90128 3340 2600 S 0.0 0.1 0:01.41 sshd
28467 root 15 0 90128 3340 2600 S 0.0 0.1 0:00.35 sshd
32602 root 16 0 90128 3332 2600 S 0.0 0.1 0:00.36 sshd
1614 root 16 0 90128 3308 2588 S 0.0 0.1 0:00.02 sshd
2817 root 5 -10 7216 3140 1724 S 0.0 0.1 0:03.80 iscsid
4161 root 15 0 66948 2340 800 S 0.0 0.0 0:10.35 sendmail
1617 nicole 17 0 53876 2000 1516 S 0.0 0.0 0:00.02 sftp-server
...
Is there anything else I should be looking at, or any more information that might be useful? I'm just a developer, but the slowdowns on this system worry me and make it hard to do my work..
Help me out, ServerFault!
