master-slave datastore replication, automatic failover, and wackamole
- by z8000
I have 2 dedicated servers provisioned for my next project's datastores. The datastores are configured for master-slave replication. There's no inherent automatic failover but I of course want this. That is, I'd love for access to the master datastore to always just work without having to configure a client library to detect when a master is down and failover to the slave.
I've seen Wackamole which is based on the Spread Toolkit. You provide Wackamole with a set of IPs and a bunch of nodes, and regardless of the up/down state of any of the nodes, those IPs will stay available/up. Wackamole detects when a node goes down and ARPs the IP(s) that were up on the now-down node. It's pretty neat actually.
So, my thought was to use Wackamole to keep the 2 virtual private IPs available/up. Clients would then just always use the same private IP to access the master datastore and the same but distinct IP for the slave datastore, even if those IPs were hosted on the same node.
My datastore servers are accessed over a private network. I am unsure if this messes with Wackamole though.
Is this lunacy? How do you generally handle automatic failover of private services like a datastore.
FWIW, it shouldn't matter but the datastore is Redis. I don't want to hear "use mySQL" please :)
Thanks.
