How can I automatically synchronize a directory tree on multiple machines?
- by Blacklight Shining
I have two Mac laptops and a Debian server, each with a directory that I would like to keep in sync between the three. The solution should meet the following criteria (in rough order of importance):
It must not use any third-party service (e.g. Dropbox, SugarSync, Google whatever). This does not include installing additional software (as long as it's free).
It must not require me to use specific directories or change my way of storing things. (Dropbox does this IIRC)
It must work in all directions (changes made on /any/ machine should be pushed to the others)
All data sent must be encrypted (I have ssh keypairs set up already)
It must work even when not all machines are available (changes should be pushed to a machine when it comes back online)
It must work even when the /directories/ on some machines are not available (they may be stored on disk images which will not always be mounted)
This can be solved for Macs by using launchd to automatically launch and kill (or in some way change the behavior of) whatever daemon is used for syncing when the images are mounted and unmounted.
It must be immediate (using an event-based system, not a periodic one like cron)
It must be flexible (if more machines are added, I should be able to incorporate them easily)
I also have some preferences that I would like to be fulfilled, but do not have to be:
It should notify me somehow if there are conflicts or other errors.
It should recognize symbolic and hard links and create corresponding ones.
It should allow me to create a list of exceptions (subdirectories which will not be synced at all).
It should not require me to set up port forwarding or otherwise reconfigure a network.
This can be solved by using an ssh tunnel with reverse port forwarding.
If you have a solution that meets some, but not all of the criteria, please contribute it in the comments as it might be useful in some way, and it might be possible to meet some of the criteria separately.
What I tried, and why it didn't work:
rsync and lsyncd do not support bidirectional synchronization
csync2 is designed for server clusters and does not appear to work with machines with dynamic IPs
DRBD (suggested by amotzg) involves installing a kernel module and does not appear to work on systems running OS X
