I'm investigating the virtual memory configuration of a Debian based Amazon EC2 instance, and as my background isn't in system admin, I'm slightly confused by what I'm seeing.
We're using MongoDB, and the monitoring server we have indicates that the Mongo process is using about 20GB of swap space, however I can't figure out where this is located on the server.
As far as I can tell from using the various suggested methods from Google, there is either a much smaller amount, or none at all.
top indicates that there is 1.8GB of swap memory:
top - 15:35:21 up 6 days, 3:23, 1 user, load average: 1.60, 1.43, 1.37
Tasks: 47 total, 2 running, 45 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 1.3%sy, 0.0%ni, 14.7%id, 83.8%wa, 0.0%hi, 0.0%si, 0.1%st
Mem: 3928924k total, 2855572k used, 1073352k free, 640564k buffers
Swap: 0k total, 0k used, 0k free, 1887788k cached
swapon -s doesn't seem to think there's any swap space:
Filename Type Size Used Priority
free -m doesn't think there's any swap either:
total used free shared buffers cached
Mem: 3836 3663 172 0 626 2701
-/+ buffers/cache: 336 3500
Swap: 0 0 0
And neither does vmstat:
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 3 0 66224 641372 2874744 0 0 21 5012 21 33 2 2 76 19
But cat /etc/fstab thinks there is a swap partition:
/dev/xvda1 / ext3 defaults 1 1
/dev/xvda2 /mnt ext3 defaults 0 0
/dev/xvda3 swap swap defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0
However df -k gives no indication of the xvda3 partition:
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda1 16513960 15675324 0 100% /
tmpfs 1964460 8 1964452 1% /lib/init/rw
udev 1914148 28 1914120 1% /dev
tmpfs 1964460 4 1964456 1% /dev/shm
So I really don't know what to make of this, because I appear to have a process using about 10 times more virtual memory than what might be available, and I have no idea where this virtual memory is on the system.
I'm probably misinterpreting the output of the tools, so I'd be grateful if someone would be able to set me straight: What have I got wrong, what's the right interpretation, and how do you reach that interpretation?
EDIT0:
We use 10gen's MMS for monitoring the database, the relevant section for memory from the last data point is:
"mem": {
"virtual": 20749,
"bits": 64,
"supported": true,
"mappedWithJournal": 20376,
"mapped": 10188,
"resident": 1219
},
This JSON is specific to the database process (I believe) rather than the system as a whole.
fdisk -l /dev/xvda outputs... nothing? I tried each of the 3 xvda entries in /etc/fstab as well:
root@ip:~# fdisk -l /dev/xvda1
Disk /dev/xvda1: 34.4 GB, 34359738368 bytes
255 heads, 63 sectors/track, 4177 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/xvda1 doesn't contain a valid partition table
root@ip:~# fdisk -l /dev/xvda2
root@ip:~# fdisk -l /dev/xvda3
root@ip:~#
Edit1:
Output of cat /proc/meminfo for the sake of completeness:
MemTotal: 3928924 kB
MemFree: 726600 kB
Buffers: 648368 kB
Cached: 2216556 kB
SwapCached: 0 kB
Active: 1945100 kB
Inactive: 994016 kB
Active(anon): 60476 kB
Inactive(anon): 12952 kB
Active(file): 1884624 kB
Inactive(file): 981064 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 387180 kB
Writeback: 0 kB
AnonPages: 73380 kB
Mapped: 1188260 kB
Shmem: 48 kB
Slab: 149768 kB
SReclaimable: 146076 kB
SUnreclaim: 3692 kB
KernelStack: 1104 kB
PageTables: 16096 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 1964460 kB
Committed_AS: 305572 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 16760 kB
VmallocChunk: 34359721448 kB
HardwareCorrupted: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 3932160 kB
DirectMap2M: 0 kB
