ZooKeeper and RabbitMQ/Qpid together - overkill or a good combination?

Posted by Chris Sears on Stack Overflow See other posts from Stack Overflow or by Chris Sears
Published on 2010-04-19T17:39:27Z Indexed on 2010/04/19 17:43 UTC
Read the original article Hit count: 343

Greetings,

I'm evaluating some components for a multi-data center distributed system. We're going to be using message queues (via either RabbitMQ or Qpid) so agents can make asynchronous requests to other agents without worrying about addressing, routing, load balancing or retransmission.

In many cases, the agents will be interacting with components that were not designed for highly concurrent access, so locking and cross-agent coordination will be needed to avoid race conditions. Also, we'd like the system to automatically respond to agent or data center failures.

With the above use cases in mind, ZooKeeper seemed like it might be a good fit. But I'm wondering if trying to use both ZK and message queuing is overkill. It seems like what Zookeeper does could be accomplished by my own cluster manager using AMQP messaging, but that would be hard to get really right. On the other hand, I've seen some examples where ZooKeeper was used to implement message queuing, but I think RabbitMQ/Qpid are a more natural fit for that.

Has anyone out there used a combination like this?

Thanks in advance,

-Chris

© Stack Overflow or respective owner

Related posts about zookeeper

Related posts about message-queue