Add entire 300 GB filesystem to Git Annex repository?
- by Ryan Lester
By default, I get an error that I have too many open files from the process. If I lift the limit manually, I get an error that I'm out of memory. For whatever reason, it seems that Git Annex in its current state is not optimised for this sort of task (adding thousands of files to a repository at once).
As a possible solution, my next thought was to do something like:
cd /
find . -type d | git annex add --$NONRECURSIVELY
find . -type f | git annex add
# Need to add parent directories of each file first or adding files fails
The problem with this solution is that there doesn't seem from the documentation to be a way to non-recursively add a directory in Git Annex. Is there something I'm missing or a workaround for this?
If my proposed solution is a dead end, are there other ways that people have solved this problem?
