optimal folder structure for storing 100k files on a USB drive
- by cherouvim
I need to store 100k files (around 40GB) in a USB drive. Each file has a unique int id (e.g 45000).
Option one is to put all files in a single folder:
root/
root/1.pdf
root/2.pdf
root/3.pdf
...
root/567.pdf
root/568.pdf
root/569.pdf
...
root/10001.pdf
root/10002.pdf
root/10003.pdf
...
root/99998.pdf
root/99999.pdf
root/100000.pdf
Option two is to create a [1-9][0-9]* folder hierarchy based on that id:
root/
root/1/file.pdf
root/2/file.pdf
root/3/file.pdf
...
root/5/6/7/file.pdf
root/5/6/8/file.pdf
root/5/6/9/file.pdf
...
root/1/0/0/0/1/file.pdf
root/1/0/0/0/2/file.pdf
root/1/0/0/0/3/file.pdf
...
root/9/9/9/9/8/file.pdf
root/9/9/9/9/9/file.pdf
root/1/0/0/0/0/0/file.pdf
Which option will scale better? I can understand that the second option will require tons of folders but each folder will at most contain 10 folders and 1 file. Maintenance will not be an issue since everything will be controlled by an application.
Note that this is a USB drive on linux and based on the above I'd also like to know whether I should go with FAT32 or NTFS.
