High memory usage on the server - can't determine the process

Posted by HTF on Server Fault See other posts from Server Fault or by HTF
Published on 2012-09-01T08:15:57Z Indexed on 2012/09/01 9:39 UTC
Read the original article Hit count: 283

Filed under:
|

I've noticed high memory usage on the server. Details:

OS: CentOS 6.3 - x86_64

Web server: Nginx with PHP-FPM

The server is generating PDF documents so the traffic is minimum.

top:

# top -b -n 1 -a
top - 10:04:51 up 21 days, 18:57,  1 user,  load average: 0.00, 0.00, 0.00
Tasks:  92 total,   1 running,  91 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.2%sy,  0.0%ni, 99.6%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   3923092k total,  3720380k used,   202712k free,   133904k buffers
Swap:  4194296k total,       12k used,  4194284k free,   147404k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                     
15855 www-data  20   0  199m 4952 2128 S  0.0  0.1   0:00.06 php-fpm                                                                                     
15853 www-data  20   0  199m 4940 2028 S  0.0  0.1   0:00.06 php-fpm                                                                                     
15850 www-data  20   0  199m 4928 2020 S  0.0  0.1   0:00.05 php-fpm                                                                                     
15851 www-data  20   0  199m 4888 2020 S  0.0  0.1   0:00.06 php-fpm                                                                                     
15852 www-data  20   0  199m 4852 2020 S  0.0  0.1   0:00.06 php-fpm                                                                                     
15857 www-data  20   0  198m 4716 2020 S  0.0  0.1   0:00.06 php-fpm                                                                                     
17553 root      20   0 97816 3860 2924 S  0.0  0.1   0:00.03 sshd                                                                                        
15849 root      20   0  198m 3460 1072 S  0.0  0.1   0:00.12 php-fpm                                                                                     
13441 nginx     20   0 65608 2968 1604 S  0.0  0.1   0:02.06 nginx                                                                                       
13440 nginx     20   0 65608 2964 1600 S  0.0  0.1   0:01.87 nginx                                                                                       
17561 root      20   0  105m 1944 1488 S  0.0  0.0   0:00.01 bash                                                                                        
 1150 xfs       20   0 20980 1784  704 S  0.0  0.0   0:00.13 xfs                                                                                         
15863 root      20   0  179m 1424 1028 S  0.0  0.0   0:00.00 rsyslogd                                                                                    
    1 root      20   0 19224 1360 1088 S  0.0  0.0   0:17.96 init                                                                                        
 1201 nrpe      20   0 40928 1288  704 S  0.0  0.0   3:57.64 nrpe                                                                                        
13226 root      20   0  114m 1216  612 S  0.0  0.0   0:00.01 crond                                                                                       
 6691 root      20   0 64068 1156  488 S  0.0  0.0   0:09.59 sshd                                                                                        
13439 root      20   0 65104 1128  292 S  0.0  0.0   0:00.00 nginx                                                                                       
19026 root      20   0 15040 1116  844 R  0.0  0.0   0:00.00 top                                                                                         
  451 root      16  -4 11052 1096  316 S  0.0  0.0   0:00.02 udevd                                                                                       
 1174 root      18  -2 11048 1064  288 S  0.0  0.0   0:00.00 udevd                                                                                       
 1175 root      18  -2 11048 1064  288 S  0.0  0.0   0:00.00 udevd                                                                                       
 1065 root      16  -4 93168  824  560 S  0.0  0.0   0:16.00 auditd                                                                                      
 1165 root      20   0  4056  564  480 S  0.0  0.0   0:00.00 mingetty                                                                                    
 1167 root      20   0  4056  564  480 S  0.0  0.0   0:00.00 mingetty                                                                                    
 1169 root      20   0  4056  564  480 S  0.0  0.0   0:00.00 mingetty                                                                                    
 1171 root      20   0  4056  564  480 S  0.0  0.0   0:00.00 mingetty                                                                                    
 1163 root      20   0  4056  560  480 S  0.0  0.0   0:00.00 mingetty                                                                                    
 1176 root      20   0  4056  560  480 S  0.0  0.0   0:00.00 mingetty                                                                                    
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd                                                                                    
    3 root      RT   0     0    0    0 S  0.0  0.0   0:11.75 migration/0                                                                                 
    4 root      20   0     0    0    0 S  0.0  0.0  44:30.28 ksoftirqd/0                                                                                 
    5 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0                                                                                 
    6 root      RT   0     0    0    0 S  0.0  0.0   0:03.51 watchdog/0                                                                                  
    7 root      RT   0     0    0    0 S  0.0  0.0   0:11.63 migration/1                                                                                 
    8 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/1                                                                                 
    9 root      20   0     0    0    0 S  0.0  0.0  11:35.50 ksoftirqd/1                                                                                 
   10 root      RT   0     0    0    0 S  0.0  0.0   0:03.34 watchdog/1                                                                                  
   11 root      20   0     0    0    0 S  0.0  0.0   1:36.68 events/0                                                                                    
   12 root      20   0     0    0    0 S  0.0  0.0   1:50.57 events/1                                                                                    
   13 root      20   0     0    0    0 S  0.0  0.0   0:00.00 cgroup                                                                                      
   14 root      20   0     0    0    0 S  0.0  0.0   0:00.00 khelper                                                                                     
   15 root      20   0     0    0    0 S  0.0  0.0   0:00.00 netns                                                                                       
   16 root      20   0     0    0    0 S  0.0  0.0   0:00.00 async/mgr                                                                                   
   17 root      20   0     0    0    0 S  0.0  0.0   0:00.00 pm                                                                                          
   18 root      20   0     0    0    0 S  0.0  0.0   0:07.86 sync_supers                                                                                 
   19 root      20   0     0    0    0 S  0.0  0.0   0:10.38 bdi-default                                                                                 
   20 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kintegrityd/0                                                                               
   21 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kintegrityd/1                                                                               
   22 root      20   0     0    0    0 S  0.0  0.0   0:04.35 kblockd/0                                                                                   
   23 root      20   0     0    0    0 S  0.0  0.0   0:04.18 kblockd/1                                                                                   
   24 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kacpid                                                                                      
   25 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kacpi_notify                                                                                
   26 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kacpi_hotplug                                                                               
   27 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata/0                                                                                       
   28 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata/1                                                                                       
   29 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata_aux                                                                                     
   30 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ksuspend_usbd                                                                               
   31 root      20   0     0    0    0 S  0.0  0.0   0:00.00 khubd                                                                                       
   32 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kseriod                                                                                     
   33 root      20   0     0    0    0 S  0.0  0.0   0:00.00 md/0                                                                                        
   34 root      20   0     0    0    0 S  0.0  0.0   0:00.00 md/1                                                                                        
   35 root      20   0     0    0    0 S  0.0  0.0   0:00.00 md_misc/0                                                                                   
   36 root      20   0     0    0    0 S  0.0  0.0   0:00.00 md_misc/1                                                                                   
   37 root      20   0     0    0    0 S  0.0  0.0   0:00.48 khungtaskd                                                                                  
   38 root      20   0     0    0    0 S  0.0  0.0   1:07.52 kswapd0                                                                                     
   39 root      25   5     0    0    0 S  0.0  0.0   0:00.00 ksmd                                                                                        
   40 root      39  19     0    0    0 S  0.0  0.0   0:22.00 khugepaged                                                                                  
   41 root      20   0     0    0    0 S  0.0  0.0   0:00.00 aio/0                                                                                       
   42 root      20   0     0    0    0 S  0.0  0.0   0:00.00 aio/1                                                                                       
   43 root      20   0     0    0    0 S  0.0  0.0   0:00.00 crypto/0                                                                                    
   44 root      20   0     0    0    0 S  0.0  0.0   0:00.00 crypto/1                                                                                    
   49 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthrotld/0                                                                                  
   50 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthrotld/1                                                                                  
   52 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kpsmoused                                                                                   
   53 root      20   0     0    0    0 S  0.0  0.0   0:00.00 usbhid_resumer                                                                              
   83 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kstriped                                                                                    
  233 root      20   0     0    0    0 S  0.0  0.0   0:00.00 scsi_eh_0                                                                                   
  234 root      20   0     0    0    0 S  0.0  0.0   0:00.00 scsi_eh_1                                                                                   
  321 root      20   0     0    0    0 S  0.0  0.0   0:00.00 virtio-blk                                                                                  
  359 root      20   0     0    0    0 S  0.0  0.0   0:03.24 kdmflush                                                                                    
  360 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kdmflush                                                                                    
  380 root      20   0     0    0    0 S  0.0  0.0   0:20.64 jbd2/dm-0-8                                                                                 
  381 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ext4-dio-unwrit                                                                             
  382 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ext4-dio-unwrit                                                                             
  694 root      20   0     0    0    0 S  0.0  0.0   0:00.00 vballoon                                                                                    
  697 root      20   0     0    0    0 S  0.0  0.0   0:00.00 virtio-net                                                                                  
  818 root      20   0     0    0    0 S  0.0  0.0   0:00.00 jbd2/vda1-8                                                                                 
  819 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ext4-dio-unwrit                                                                             
  820 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ext4-dio-unwrit                                                                             
  851 root      20   0     0    0    0 S  0.0  0.0   0:06.96 kauditd                                                                                     
 1013 root      20   0     0    0    0 S  0.0  0.0   0:15.45 flush-253:0

ps:

# ps aux --sort -vsz | head
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
www-data 13213  0.0  0.1 204416  4772 ?        S    08:28   0:00 php-fpm: pool default
www-data 13214  0.0  0.1 204416  4776 ?        S    08:28   0:00 php-fpm: pool default
www-data 13215  0.0  0.1 204416  4832 ?        S    08:28   0:00 php-fpm: pool default
www-data 13216  0.0  0.1 204416  4776 ?        S    08:28   0:00 php-fpm: pool default
www-data 13218  0.0  0.1 204416  4956 ?        S    08:28   0:00 php-fpm: pool default

free:

#free -m
             total       used       free     shared    buffers     cached
Mem:          3831       3530        300          0        130        143
-/+ buffers/cache:       3256        574
Swap:         4095          0       4095

When I stooped Nginx, PHP-FPM the memory usage was still the same.

Could you help me to investigate what is consuming the memory on the system?

Regards

© Server Fault or respective owner

Related posts about nginx

Related posts about php-fpm