Introduction to messaging queue – RabbitMQ

07 May 2021

Back in time to the days of monolithic architecture. Back then application components were tightly coupled. That means they were directly connected. So in a simple retail application, if we had checkout service, and it needed to communicate with an inventory service, that would be done directly through a TCP connection. So this had some limitations. As soon as the checkout sends the message it would need to hear a reply before it can move on to the next task, or worse if the inventory service went down it would try over and over again until it was able to make that connection.

So that’s why message queues were created or message brokers, and those will sit, a message queue sits in between the two services that need to communicate with one another. So with a message queue, a checkout can add a message to the queue and then immediately move on to the next task, and then similarly the inventory when it’s ready can consume from the queue, process the message, and then immediately consume the next message. So this is going to decouple the two applications.

A message broker is also going to help with scalability. If a lot of checkouts happen at once, the queue begins to fill, we can have more than one consuming service to read from the queue to handle the amount of workload that checkout is producing, and that’s going to make the system more scalable. Another benefit of message queues is that the queue itself can sit on its own machine that allows it to offload some of the work that’s done by the web application and make the whole system more efficient.

So what is RabbitMQ

Rabbit MQ is an implementation of the AMQP message model, that is an advanced message queuing protocol. With this type of message model the producer, in our case the checkout (the service that produces the messages) instead of producing directly to a message queue it’s going to produce to an exchange. We can think of the exchange as a post office.

Exchange is going to receive all the messages and then distribute them accordingly to how they’re addressed. An exchange could be connected to many different queues, and then queues are connected to the consuming services or consumers. We have two consuming services Inventory and shipping that need to be consumed from a checkout.

So the checkout will send a message to the exchange. An exchange is connected to queues through connections called bindings, and these bindings can be referenced by binding keys, and then our consuming services, or applications, subscribe to the queues.

One thing that’s great about this message model is the flexibility with which messages can move through the system, and that flexibility is largely in part to the different types of exchanges available.

Types of exchanges

• Fanout Exchange: With a fanout exchange checkout will produce an exchange, the exchange will duplicate the message and send it to every single queue that it knows about.
• Direct Exchange: With a direct exchange, the checkout will produce a message and then that message will get a routing key. So with direct exchange, the routing key is being compared to the binding key, and if it’s an exact match then the message will move through the system accordingly.
• Topic Exchange: With a topic exchange we can do a partial match between the routing key and binding key.
• Header Exchange: With a header exchange the routing key is ignored completely and the message is moved through the system according to the header. It uses message header attributes for routing.
• Default Exchange: This exchange is unique only to RabbitMQ. it’s not part of the AMQP message model. It is also called nameless exchange.

The way the message moves through the system is largely a part of the message metadata. So in this case it’s the application and the developer that has a lot of control with the way the message moves through the system rather than the broker administrator.

Message flow in RabbitMQ

1. The message is published to an exchange by the producer. When creating an exchange, the type must be specified.
2. The exchange receives the message and is now responsible for routing the message. The exchange takes different message metadata into account, such as the routing key, depending on the exchange type.
3. Bindings must be created from the exchange to queues. In this case, there are two bindings to two different queues from the exchange. The exchange routes the message into the queues depending on message metadata.
4. The messages remains in the queue until they are processed by a consumer
5. The consumer processes the message.

Implementation

We will understand the concept of RabbitMQ with the help of two small programs in Javascript; a producer that sends a single message, and a consumer that receives messages and prints them out.

First, install amqp.node using npm:

Now we have amqp.node installed, we can write some code.

Sending:

In send.js, we need to require the library first and then connect to RabbitMQ server.

To send, we must declare a queue for us to send to; then we can publish a message to the queue. Declaring a queue is idempotent – it will only be created if it doesn’t exist already. The message content is a byte array, so you can encode whatever you like there.

Lastly, we close the connection and exit:

Receiving:

Our consumer services listens for messages from RabbitMQ server, so unlike the publisher which sends a single message, we’ll keep the consumer  running to listen for messages and process them out.

Set up is the same as the publisher; here we will open a connection and a channel, and declare the queue(same queue where we have pushed) from which we’re going to consume.

We will tell the server to deliver messages from the queue. Since it will push messages asynchronously, we wrote a callback that will be executed when RabbitMQ pushes messages to our consumer.

Conclusion

RabbitMQ can do cross language communication and it is a lightweight, reliable, scalable and portable message broker. Message queues enable different parts of a system to communicate and performs operations asynchronously. A message queue provides a lightweight buffer which temporarily stores messages, and endpoints that allow software components to connect to the queue in order to send and receive messages. In order to do that, the broker uses two key components: exchanges and queues