Why is domU faster than dom0 on IO?
- by Paco
I have installed debian 7 on a physical machine. This is the configuration of the machine:
3 hard drives using RAID 5
Strip element size: 1M
Read policy: Adaptive read ahead
Write policy: Write Through
/boot 200 MB ext2
/ 15 GB ext3
SWAP 10GB
LVM rest (~500GB)
emphasized text
I installed postgresql, created a big database (over 1GB). I have an SQL request that takes a lot of time to run (a SELECT statement, so it only reads data from the database). This request takes approximately 5.5 seconds to run.
Then, I installed XEN, created a domU, with another debian distro. On this OS, I also installed postgresql, with the same database. The same SQL request takes only 2.5 seconds to run.
I checked the kernel on both dom0 and domU. uname-a returns "Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2+deb7u2 x86_64 GNU/Linux" on both systems.
I checked the kernel parameters, which are approximately the same. For those that are relevant, I changed their values to make them match on both systems using sysctl. I saw no changes (the requests still take the same amount of time).
After this, I checked the file systems. I used ext3 on domU. Still no changes.
I installed hdparm, and ran hdparm -Tt on both systems, on all my partitions on both systems, and I get similar results.
Now, I am stuck, I don't know what is different, and what could be the cause of such a big difference.
Additional Info:
Debian runs on a Dell server PowerEdge 2950
postgresql: 9.1.9 (both dom0 and domU)
xen-linux-system: 3.2.0
xen-hypervisor: 4.1
Thanks
EDIT:
As Krzysztof Ksiezyk suggested, it might be due to some file caching system. I ran the dd command to test both the read and write speed.
Here is domU:
root@test1:~# dd if=/dev/zero of=/root/dd count=5MB bs=1MB
^C2020+0 records in
2020+0 records out
2020000000 bytes (2.0 GB) copied, 18.8289 s, 107 MB/s
root@test1:~# dd if=/root/dd of=/dev/null count=5MB bs=1MB
2020+0 records in
2020+0 records out
2020000000 bytes (2.0 GB) copied, 15.0549 s, 134 MB/s
And here is dom0:
root@debian:~# dd if=/dev/zero of=/root/dd count=5MB bs=1MB
^C1693+0 records in
1693+0 records out
1693000000 bytes (1.7 GB) copied, 8.87281 s, 191 MB/s
root@debian:~# dd if=/root/dd of=/dev/null count=5MB bs=1MB
1693+0 records in
1693+0 records out
1693000000 bytes (1.7 GB) copied, 0.501509 s, 3.4 GB/s
What can be the cause of this caching system? And how can we "fix" it? Can we apply it to dom0?
EDIT 2:
I switched my virtual disk type. To do so I followed this article.
I did a dd if=/dev/vg0/test1-disk of=/mnt/test1-disk.img bs=16M
Then in /etc/xen/test1.cfg, I changed the disk parameter to use file: instead of phy:
it should have removed the file caching, but I still get the same numbers (domU being much faster for Postgres)
