How to keep subtree removal (`rm -rf`) from starving other processes for Disk I/O?
- by David Eyk
We have a very large (multi-GB) Nginx cache directory for a busy site, which we occasionally need to clear all at once. I've solved this in the past by moving the cache folder to a new path, making a new cache folder at the old path, and then rm -rfing the old cache folder.
Lately, however, when I need to clear the cache on a busy morning, the I/O from rm -rf is starving my server processes of disk access, as both Nginx and the server it fronts for are read-intensive. I can watch the load average climb while the CPUs sit idle and rm -rf takes 98-99% of Disk IO in iotop.
I've tried ionice -c 3 when invoking rm, but it seems to have no appreciable effect on the observed behavior.
Is there any way to tame rm -rf to share the disk more? Do I need to use a different technique that will take its cues from ionice?
Update:
The filesystem in question is an AWS EC2 instance store (the primary disk is EBS). The /etc/fstab entry looks like this:
/dev/xvdb /mnt auto defaults,nobootwait,comment=cloudconfig 0 2
