What is a message broker?
And
What are Dead Letter Queues?
Message Broker
Message brokers are a piece of software that facilitates inter-application communication.
Dead Letter Queue
A DLQ – Dead Letter Queue is often referred to as an undelivered message queue. It is holding a message queue for messages that cannot be delivered to the destination queues owing to multiple factors.
These factors include the non-existence of destination queues or either because these destination queues are full.
Taming Asynchronous Messages
When working with a mounting number of asynchronous messages – as long as the applications work fine without running into errors – this won’t pose any problems.
But what happens when a message doesn’t get delivered?
Occasionally messages do not get delivered.
For such cases – the basic way forward is to move the message to a DLQ. Here the message will stay dormant until the administrator is ready to take a look. But, this is only the beginning of a major problem – as the solutions are not always straightforward.
You can try out the basic and common practices – but information concerning these is difficult if not impossible to find.
So, improvisation is the key!
Option – 1
Auto Re-process Messages
The common practice is to retry and initiate a new attempt.
In this, after a number of attempts, the message gets deleted for good. Newer message broker software offers this functionality as a feature, which you might need to configure.
This strategy seems smart – in theory.
But this can also make the situation worse – as automation invites the developers not to look closely anymore, and the real errors will occur more frequently to clutter the log.
This option could work when the message delivery time is crucial, and retrying poses a good chance of guaranteeing success in message delivery.
Option – 2
Organizational Strategy
Go for a good organizational strategy that isn’t obvious.
Messages that are stuck are commonly a part of the error handling of the production system. These errors, however, cannot be anticipated – otherwise, one could and would build a solution ahead of the errors occurring.
DLQ handling is also a tiresome and insufferable topic.
Many people look the other way at the faulty messages – as the issue is technical and stays only relevant within the development team. Possibly not even getting noticed unless a customer makes a complaint.
Shared Responsibility – A no-no!
No one is responsible if everyone is made responsible; hence, choose a unique person – better if alternating – who regularly checks and determines if there is a bug. This effort can not be optimized with automation without ignoring bugs. Plus, poorer tools are mainly the reason that bugs in the first place get ignored.
Option – 3
Directly Drop Messages
You can avoid a dead letter queue altogether!
What’s that like?
In a micro-service structure, a process goes through many steps; some are stored. If communicating from one service to the next – one could regenerate a message from the database anytime.
So, why fill another dead letter queue? The microservice receiving the message can drop it in case of error and record it on the log for traceability.
This option is considerably good for recurring jobs, which monitor the database anyway. In the best-case scenario, the status is also returned.
Option – 4
Semi-Automated Handling
This might be it so pay attention.
While recognizing that manual effort is never replaced or can never be avoided, there are situations in which one could proceed in an automated way.
Therefore, a semi-automated handling solution could look like this;
Start by making sure that no messages are lost – for this, the first step is to read it out so it can stay comfortably in the dead letter queue by not getting acknowledged. Then you can begin to decide what can happen with the message.
For instance, lets say you have 3 messages, which are numbered for the sake of simplicity. After comparing and contrasting, you can make the following decisions;
- Delete message 1
- Message 2 is a bug that is on priority to fix later and can be ignored right now.
- Message 3 has broken content, but a script can be run for it, repairing the message.
Having decided on these – one can re-read messages and process them.
Here you can easily re0queue the ignored message-2 at the end of the DLQ.
Final Thoughts
When using a message broker – you might need time to investigate the errors in the messages manually.
Automation can support to an extent – but this should not be termed the final solution. Also, it is recommended to avoid unnecessary dead letter messages wherever possible as the manual effort does not scale. In the end, look for improved tools that are fun to work with and take the load off the developers.
For better solutions to these common and recurring problems – check out Memphis{dev} – your efficient partner for effective inter-services communication.
0
0
Average Rating