Detecting source of memory usage on a Linux box

Posted by apeace on Super User See other posts from Super User or by apeace
Published on 2011-01-09T20:08:35Z Indexed on 2011/01/10 1:55 UTC
Read the original article Hit count: 521

I have a toy Linux box with 256mb RAM running Ubuntu 10.04.1 LTS. Here is the output of free -m:

             total       used       free     shared    buffers     cached
Mem:           245        122        122          0         19         64
-/+ buffers/cache:         38        206
Swap:          511          0        511

Unless I'm reading this wrong, 122mb is being used and only 84mb of that is disk cache. Here are all processes I'm running sorted by memory usage (ps -eo pmem,pcpu,rss,vsize,args | sort -k 1 -r):

%MEM %CPU   RSS    VSZ COMMAND
 5.0  0.0 12648 633140 node /home/node/main/sites.js
 1.5  0.0  3884 251736 /usr/sbin/console-kit-daemon --no-daemon
 1.3  0.0  3328  77108 sshd: apeace [priv]
 0.9  0.0  2344  19624 -bash
 0.7  0.0  1776  23620 /sbin/init
 0.6  0.0  1624  77108 sshd: apeace@pts/0
 0.6  0.0  1544   9940 redis-server /etc/redis/redis.conf
 0.6  0.0  1524  25848 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 103:105
 0.5  0.0  1324 119880 rsyslogd -c4
 0.4  0.0  1084  49308 /usr/sbin/sshd
 0.4  0.0  1028  44376 /usr/sbin/exim4 -bd -q30m
 0.3  0.0   904   6876 ps -eo pmem,pcpu,rss,vsize,args
 0.3  0.0   888  21124 cron
 0.3  0.0   868  23472 dbus-daemon --system --fork
 0.2  0.0   732  19624 -bash
 0.2  0.0   628   6128 /sbin/getty -8 38400 tty1
 0.2  0.0   628  16952 upstart-udev-bridge --daemon
 0.2  0.0   564  16800 udevd --daemon
 0.2  0.0   552  16796 udevd --daemon
 0.2  0.0   548  16796 udevd --daemon
 0.0  0.0     0      0 [xenwatch]
 0.0  0.0     0      0 [xenbus]
 0.0  0.0     0      0 [sync_supers]
 0.0  0.0     0      0 [netns]
 0.0  0.0     0      0 [migration/3]
 0.0  0.0     0      0 [migration/2]
 0.0  0.0     0      0 [migration/1]
 0.0  0.0     0      0 [migration/0]
 0.0  0.0     0      0 [kthreadd]
 0.0  0.0     0      0 [kswapd0]
 0.0  0.0     0      0 [kstriped]
 0.0  0.0     0      0 [ksoftirqd/3]
 0.0  0.0     0      0 [ksoftirqd/2]
 0.0  0.0     0      0 [ksoftirqd/1]
 0.0  0.0     0      0 [ksoftirqd/0]
 0.0  0.0     0      0 [ksnapd]
 0.0  0.0     0      0 [kseriod]
 0.0  0.0     0      0 [kjournald]
 0.0  0.0     0      0 [khvcd]
 0.0  0.0     0      0 [khelper]
 0.0  0.0     0      0 [kblockd/3]
 0.0  0.0     0      0 [kblockd/2]
 0.0  0.0     0      0 [kblockd/1]
 0.0  0.0     0      0 [kblockd/0]
 0.0  0.0     0      0 [flush-202:1]
 0.0  0.0     0      0 [events/3]
 0.0  0.0     0      0 [events/2]
 0.0  0.0     0      0 [events/1]
 0.0  0.0     0      0 [events/0]
 0.0  0.0     0      0 [crypto/3]
 0.0  0.0     0      0 [crypto/2]
 0.0  0.0     0      0 [crypto/1]
 0.0  0.0     0      0 [crypto/0]
 0.0  0.0     0      0 [cpuset]
 0.0  0.0     0      0 [bdi-default]
 0.0  0.0     0      0 [async/mgr]
 0.0  0.0     0      0 [aio/3]
 0.0  0.0     0      0 [aio/2]
 0.0  0.0     0      0 [aio/1]
 0.0  0.0     0      0 [aio/0]

Now, I know that ps is not the best for viewing process memory usage, but that's because it tends to report more memory than is actually being used...meaning no matter how you look at it all my processes combined shouldn't be using near 122mb, even if you account for the disk cache.

What's more, memory usage is growing all the time. I've had to restart my server once a week, because once my 256mb fills up it starts swapping, which it wouldn't do just for disk cache. Shouldn't there be some way for me to see the culprit?!

I'm new to server admin, so please if there's something obvious I'm overlooking point it out to me.

Just for good measure, the output of cat /proc/meminfo:

MemTotal:         251140 kB
MemFree:          124604 kB
Buffers:           20536 kB
Cached:            66136 kB
SwapCached:            0 kB
Active:            65004 kB
Inactive:          37576 kB
Active(anon):      15932 kB
Inactive(anon):      164 kB
Active(file):      49072 kB
Inactive(file):    37412 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:        524284 kB
SwapFree:         524284 kB
Dirty:                 8 kB
Writeback:             0 kB
AnonPages:         15916 kB
Mapped:            10668 kB
Shmem:               188 kB
Slab:              18604 kB
SReclaimable:      10088 kB
SUnreclaim:         8516 kB
KernelStack:         536 kB
PageTables:         1444 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      649852 kB
Committed_AS:      64224 kB
VmallocTotal:   34359738367 kB
VmallocUsed:         752 kB
VmallocChunk:   34359737600 kB
DirectMap4k:      262144 kB
DirectMap2M:           0 kB

EDIT: I had misinterpreted the meaning of free -m at first. But even so: the important thing is that my OS eventually begins to use swap memory if I don't restart my server, which disk caching wouldn't do. So where do I look to see what is using all this memory?

© Super User or respective owner

Related posts about linux

Related posts about memory