What's a good tool for collecting statistics on filesystem usage?

Posted by Kamil Kisiel on Server Fault See other posts from Server Fault or by Kamil Kisiel
Published on 2009-06-09T17:51:28Z Indexed on 2012/10/31 17:03 UTC
Read the original article Hit count: 163

Filed under:

linux

|

filesystems

We have a number of filesystems for our computational cluster, with a lot of users that store a lot of really large files. We'd like to monitor the filesystem and help optimize their usage of it, as well as plan for expansion.

In order to this, we need some way to monitor how these filesystems are used. Essentially I'd like to know all sorts of statistics about the files:

Age
Frequency of access
Last accessed times
Types
Sizes

Ideally this information would be available in aggregate form for any directory so that we could monitor it based on project or user.

Short of writing something up myself in Python, I haven't been able to find any tools capable of performing these duties. Any recommendations?

© Server Fault or respective owner

Related posts about linux

apt-get install and update fail

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've got a problem with apt-get update and apt-get install ... commands . every time update or installing fails and errors are : Get:1 http://dl.google.com stable Release.gpg [198B] Ign http://dl.google.com/linux/chrome/deb/ stable/main Translation-en_US Get:2 http://dl… >>> More
kernel module compiling error

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
sh@ubuntu:/home/ccpp/helloworld$ make gcc-4.6 -O2 -DMODULE -D_KERNEL_ -W -Wall -Wstrict-prototypes -Wmissing-prototypes -isystem /lib/modules/`uname -r`/build/include -c -o hello-1.o hello-1.c hello-1.c:4:0: warning: "MODULE" redefined [enabled by default] <command-line>:0:0: note: this is… >>> More
Build-Essentials installation failing

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am having trouble accessing the several critical header files that show to be a part of the build process. The "Ubuntu Software Center" shows "Build Essentials" as installed: Next I did the following two commands, which did not improve the problem: ~$ sudo apt-get install build-essential [sudo]… >>> More
Updating Debian kernel

as seen on Super User - Search for 'Super User'
I'm trying to update my Debian machine to 2.6.32-46 (which is the new stable). However, after doing apt-get update my apt-cache search linux-image shows me: linux-headers-2.6.32-5-486 - Header files for Linux 2.6.32-5-486 linux-headers-2.6.32-5-686-bigmem - Header files for Linux 2.6.32-5-686-bigmem linux-headers-2… >>> More
Serial connection over a single USB cable (Windows to linux, or linux to linux)

as seen on Server Fault - Search for 'Server Fault'
I'm helping out with a project for an embedded device that only has USB and no serial. This device is running Linux. These days, when we need to connect to a serial port on a device we typically use a USB to serial adapter (on something like a phone system or a load balancing device, etc). I would… >>> More

Related posts about filesystems

Question regarding filesystems true or false?

as seen on Super User - Search for 'Super User'
Hello all, though I'm familiar with stackoverflow , and loving it , i've actually got a couple of questions myself about something other then programming. Here are my question Is it true that in FAT filesystems the maximum number of files per filesystem equals the number of entries in the FAT table… >>> More
Filesystems for webserver with SATA and Solid State disk,

as seen on Server Fault - Search for 'Server Fault'
We have just ordered a new webserver with 120 Gb solid state disk and a SATA disk. I am trying to plan ahead what sort of filesystem to use. This system will be running Linux, Apache/Tomcat to host java services. The main service is a system where people can upload reasonably large files (in the… >>> More
To see virtual filesystems in Ubuntu

as seen on Super User - Search for 'Super User'
How can you see the virtual filesystems at root? They should be /proc and /dev. >>> More
How do filesystems handle concurrent read/write?

as seen on Stack Overflow - Search for 'Stack Overflow'
User A asks the system to read file foo and at the same time user B wants to save his or her data onto the same file. How is this situation handled on the filesystem level? >>> More
Strange ZFS hidden filesystem problem

as seen on Server Fault - Search for 'Server Fault'
Half of my ZFS filesystems are hidden in ZFS-fuse. Here's my story: So, I love ZFS. I used it for about six months on FreeBSD, but due to it crashing the kernel during heavy inter-filesystem IO load, I tried switching to Solaris 5.10. That was good, but when I attempted to do an import of my Version… >>> More