I'm experiencing some significant performance issues on an NFS server. I've been reading up a bit on partition alignment, and I think I have my partitions mis-aligned. I can't find anything that tells me how to actually quantify the effects of mis-aligned partitions. Some of the general information I found suggests the performance penalty can be quite high (upwards of 60%) and others say it's negligible. What I want to do is determine if partition alignment is a factor in this server's performance problems or not; and if so, to what degree?
So I'll put my info out here, and hopefully the community can confirm if my partitions are indeed mis-aligned, and if so, help me put a number to what the performance cost is.
Server is a Dell R510 with dual E5620 CPUs and 8 GB RAM. There are eight 15k 2.5” 600 GB drives (Seagate ST3600057SS) configured in hardware RAID-6 with a single hot spare. RAID controller is a Dell PERC H700 w/512MB cache (Linux sees this as a LSI MegaSAS 9260). OS is CentOS 5.6, home directory partition is ext3, with options “rw,data=journal,usrquota”.
I have the HW RAID configured to present two virtual disks to the OS: /dev/sda for the OS (boot, root and swap partitions), and /dev/sdb for a big NFS share:
[root@lnxutil1 ~]# parted -s /dev/sda unit s print
Model: DELL PERC H700 (scsi)
Disk /dev/sda: 134217599s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 63s 465884s 465822s primary ext2 boot
2 465885s 134207009s 133741125s primary lvm
[root@lnxutil1 ~]# parted -s /dev/sdb unit s print
Model: DELL PERC H700 (scsi)
Disk /dev/sdb: 5720768639s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 34s 5720768606s 5720768573s lvm
Edit 1
Using the cfq IO scheduler (default for CentOS 5.6):
# cat /sys/block/sd{a,b}/queue/scheduler
noop anticipatory deadline [cfq]
noop anticipatory deadline [cfq]
Chunk size is the same as strip size, right? If so, then 64kB:
# /opt/MegaCli -LDInfo -Lall -aALL -NoLog
Adapter #0
Number of Virtual Disks: 2
Virtual Disk: 0 (target id: 0)
Name:os
RAID Level: Primary-6, Secondary-0, RAID Level Qualifier-3
Size:65535MB
State: Optimal
Stripe Size: 64kB
Number Of Drives:7
Span Depth:1
Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteThrough, ReadAdaptive, Direct, No Write Cache if Bad BBU
Access Policy: Read/Write
Disk Cache Policy: Disk's Default
Number of Spans: 1
Span: 0 - Number of PDs: 7
... physical disk info removed for brevity ...
Virtual Disk: 1 (target id: 1)
Name:share
RAID Level: Primary-6, Secondary-0, RAID Level Qualifier-3
Size:2793344MB
State: Optimal
Stripe Size: 64kB
Number Of Drives:7
Span Depth:1
Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteThrough, ReadAdaptive, Direct, No Write Cache if Bad BBU
Access Policy: Read/Write
Disk Cache Policy: Disk's Default
Number of Spans: 1
Span: 0 - Number of PDs: 7
If it's not obvious, virtual disk 0 corresponds to /dev/sda, for the OS; virtual disk 1 is /dev/sdb (the exported home directory tree).
