MQTT Publish Subscribe

There are two design patterns that messaging systems follow: Message Queues and Publish-Subscribe.

• Message Queues

This message queue pattern is designed for scenarios like a task list or work queue where each message stored in it must be read and processed only once by any one consumer. It is sometimes referred to as point-to-point messaging. Generally, in this pattern, the consumers are multiple instances of the same application that do the same processing. Let’s take the example of an online image processing service. Multiple users will upload images to be processed (the messages in this case), at the same time using their web browsers (the producers in this case). The image processing service needs to process these uploaded images (resize, change colors, convert formats, etc.), which could take some time.

Instead of making a client (the browser and the user) wait for the processing to complete, images are received and held in a queue, making them waiting to be processed. The users receive an acknowledgment that their uploaded image has been received and will be processed shortly, so they can move on to doing something else on their browser and check back later to view their uploaded image. The consumer - the image processing application, will read the first message in the queue (the image data in this case), process it, and post the processed image to a location from which the browser can access it. After that, it will read the next message in the queue, process it and post it, and so on, until all messages (images) present in a queue are processed. In order to increase the processing capacity, we can also add multiple consumers, where each of them will read the next message, which is waiting in the queue, and process it. Since an image should not be processed more than once, only one consumer should receive each message. In order to ensure such, each message present in a queue will be deleted once read by a consumer. This pattern works well as a load balancing system since messages (or work items) are picked up by the next available consumer, and all consumers are equally loaded.

• Publish-Subscribe In the publish-subscribe (pub-sub) pattern, one message can be read by multiple consumers. It is a one-to-many pattern, where, unlike in the message queue model, it does not matter if the message is processed multiple times. It is even possible that a message may be processed in different ways by each consumer. Here a category is referred to as a Topic for a particular type of message and is created in the message broker. Depending on the message, the producer published it to a topic. Consumers can subscribe to a topic, and each one of them will receive all the messages published on that topic. One topic can have multiple consumers, and one consumer can subscribe to various topics, where both patterns can be used in an IoT solution. Where vast amounts of data are read by sensors and sent for processing, a message queue pattern with multiple consumers can be used to process the data.

Alternately, where processing speed may not be important, the Pub-Sub pattern can be used. Each type of sensor will publish its data on a topic. The consumers, whether a smartphone app, a Node-RED flow, or any other process, will read the sensor data that they are interested in by subscribing to the relevant topics. Each consumer may subscribe to one or more topics, and it will receive all the messages published to that topic. While both models can be used in any use case, there are some advantages and constraints to each model; thus, they must be used judiciously depending on the requirements of the solution.