Buy @ Amazon

RabbitMQ System Overview - A Cheatsheet

on
There are a plethora of Message Brokers or Queuing systems, each with it's own set of nuances that come from how it is designed to work. For someone like me who dabbles on varied technologies it gets really hard to remember some key details. This is a cheatsheet of sorts that I had prepared to quickly refer to, for reflection, introspection and analysis of current system design that employs RabbitMQ in its Architecture. 

Whether you are a Developer, or an Architect or an Engineering Leader, this should help you get your understanding right about how RabbitMQ works and the components involved in its ecosystem.

Some Key Pointers

  1. Every Queue is a RabbitMQ process.
  2. Queues are not replicated by design.
  3. Supported protocols include AMQP, MQTT, STOMP and HTTP.
  4. Conceptually, you can have, "Many Publishers to a Queue". Many Publishers like ProApp1, ProApp2, .., ProAppN can publish message to a same queue.
  5. Quest For Design of Queues :
    1. Approach 1 - A Queue Per Subscriber App/Service : No more than one group of Consumer Apps should subscribe to the same queue, i.e . you can have many instance of ConApp1 subscribing to Queue1, but cannot have another app ConApp2 subscribing to same Queue1.
    2. Approach 2 - A Queue-Message Per Subscriber App/Service : This is a design situation when you have more than one kind of Subscriber App/Service mapped to a Queue but then they are not interested in the same message but different ones. For example, OrderPlacedJob services and OrderDispatchedJob services can subscribe to a queue named Order each interested in their own kind of message exclusive to it.
  6. RabbitMQ pushes messages to its dumb subscribers for their consumption.
  7. Do monitor your queues for its size. If all subscriber instances for a queue are busy, then that queue can easily fill-up and slow down your RabbitMQ. The size of a queue determines how many subscriber instances aka workers are needed to process the messages in the queue.
  8. To speed up processing, RabbitMQ may send more than 1 msg to a consumer/subscriber.
  9. Message Acknowledgement : As default, a message delivered to a consumer/subscriber is marked for deletion.
  10. No message timeouts : RabbitMQ redelivers a message when a consumer/subscriber dies.
  11. You may have to optimize Prefetch Count from time to time depending on the change in your business needs.
  12. RabbitMQ by default uses round-robin method of distributing messages in the queue to its connected subscribers/consumers/workers to achieve fair load distribution. However, if there were to be say 250 messages in a queue before any subscriber becoming active,  the first subscriber to get connected to the queue might get all the messages (depending on the prefetch count, where the default is 20 but it could have been modified in production servers for performance reasons), leaving the subsequent consumers idle until the new set of messages arrive.