Need help trouble shooting high CPU usage by PHP-fpm
- by user432506
There is a problem that is driving me crazy. After the day I tried to fix the CPU usage problem of my VPS, the CPU load has grown from 60% to 150%, and I have no idea what causes the problem. Please help me.
I had installed a copy of mediawiki on a Linode 1024. The wiki is running on Niginx + PHP-fpm + MySql. The wiki doesn't have much traffic, only around 4000 requests/day, mostly from Google and Bing bots. It had been using around 60% (of total 400% on the Linode) of the CPU before. I thought it was a bit high, so two day ago, I was trying to fix the problem (not knowing what was waiting for me). I did nothing but added a new empty line to wiki's configure file, which would change the modified time of the configure file, and then all the cached page files would be set invalidated. I had done that before, and that would cause high CPU usage, but normally it would take only hours to let things back to normal again. Not this time, my CPU usage has been around 150% for more than two days.
It is php-fpm using most of CPU reassures. Using 100% of three cores is not rare. I hadn't seen that before. There are other sites on the Linode, but it should be the wiki. Because if I offline the wiki, CPU usage will drop back to around 40% soon.
The day I also duplicated php-fpm.conf, and opened it, but didn't changed it.
I have no idea what I did wrong. I here ask for help to save myself from being crazy!!! It is php-fpm. Is there a way to find out what is it doing? I mean like which scripts are related and what function codes are running?
top:
top - 06:34:33 up 10 days, 4:23, 2 users, load average: 1.10, 1.24, 1.37
Tasks: 76 total, 4 running, 72 sleeping, 0 stopped, 0 zombie
Cpu(s): 61.1%us, 3.1%sy, 0.0%ni, 32.8%id, 2.9%wa, 0.0%hi, 0.0%si, 0.1%st
Mem: 1028684k total, 945192k used, 83492k free, 89580k buffers
Swap: 524284k total, 18084k used, 506200k free, 530380k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
26721 www-data 20 0 208m 54m 34m R 99 5.4 0:09.07 /opt/php5/sbin/php-fpm --fpm-config /opt/php5/etc/php-fpm.conf
26592 www-data 20 0 207m 45m 26m R 91 4.5 0:12.77 /opt/php5/sbin/php-fpm --fpm-config /opt/php5/etc/php-fpm.conf
26706 www-data 20 0 196m 43m 34m S 47 4.3 0:15.19 /opt/php5/sbin/php-fpm --fpm-config /opt/php5/etc/php-fpm.conf
26583 www-data 20 0 197m 45m 35m S 33 4.5 0:19.08 /opt/php5/sbin/php-fpm --fpm-config /opt/php5/etc/php-fpm.conf
26787 www-data 20 0 206m 36m 18m R 25 3.7 0:00.41 /opt/php5/sbin/php-fpm --fpm-config /opt/php5/etc/php-fpm.conf
26661 www-data 20 0 207m 46m 26m S 13 4.6 0:19.87 /opt/php5/sbin/php-fpm --fpm-config /opt/php5/etc/php-fpm.conf
1971 mysql 20 0 155m 57m 3952 S 8 5.7 383:57.81 /usr/sbin/mysqld
242 root 20 0 0 0 0 S 1 0.0 0:51.36 [kworker/3:1]
5711 root 20 0 139m 95m 580 S 1 9.5 0:41.30 /usr/local/bin/memcached -d -u root -m 128 -p 11211
19463 root 20 0 190m 3984 1284 S 1 0.4 0:02.66 /opt/php5/sbin/php-fpm --fpm-config /opt/php5/etc/php-fpm.conf
29100 www-data 20 0 10928 5540 820 S 1 0.5 4:49.05 nginx: worker process
vmstat 30
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 16912 81456 90784 554172 0 0 4 6 3 2 11 1 87 1
0 0 16912 78036 91000 555356 0 0 38 34 1397 375 12 1 87 0
4 0 16912 31776 91528 557508 0 0 78 42 3197 487 45 1 52 1
1 0 16912 83356 91768 558576 0 0 35 56 2608 449 32 1 67 1
1 0 16912 81548 92040 559720 0 0 41 31 1243 432 8 1 91 1
2 0 16912 53056 92332 562744 0 0 105 33 2013 581 17 1 81 1
2 0 16912 73236 92552 564844 0 0 68 36 1968 615 16 1 82 1
0 0 16912 91612 92904 566676 0 0 69 35 1845 692 13 1 85 1
1 0 16912 71248 93180 568428 0 0 58 33 1952 604 15 1 82 1
1 0 16868 55952 93516 572660 1 0 144 42 1801 637 12 1 86 1
2 0 16868 48324 94416 577844 0 0 189 66 2058 702 17 1 80 2
1 0 16928 58644 94592 578184 0 2 160 49 2578 723 25 1 70 3
5 0 16928 22600 94980 580568 0 0 89 32 1496 361 13 0 85 1
0 0 16988 49256 94500 576396 0 2 41 37 1601 426 14 1 85 0
5 0 18084 24336 86032 502748 0 37 83 68 2989 562 42 1 56 0
1 0 18084 123604 86376 506996 0 0 118 41 2201 573 22 1 76 1
2 0 18084 126984 86752 508876 0 0 64 53 1620 490 13 1 85 1
2 0 18084 103104 87148 510768 0 0 71 37 2757 602 33 1 64 1
