RDBMS for messaging
Is messaging in the RDBMS database a sin? Lukas Eder shares his thoughts on the topic.
This post was originally published over at jooq.org as part of a special series focusing on all things Java 8, including how take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub.
Controversial database topics are a guaranteed success on reddit, because everyone has an opinion on those topics. More importantly, many people have a dogmatic opinion, which always triggers more debate than pragmatism.
… and I’m saying: No. Messaging in the database is not an anti-pattern, it is (can be) totally OK. Let’s consider why:
KISS and YAGNI
First off, if you don’t plan on deploying thousands of message types and millions of messages per hour, then you might have a very simple messaging problem. Since you’re already using an RDBMS, using that RDBMS also for messaging is definitely an option.
Obviously, many of you will now think:
If all you have is a hammer, everything looks like a nail
… and you’re right! But guess what, the reasons for only having a hammer can be any of:
- You don’t have the time to equip with a sophisticated tool box
- You don’t have the money to equip with a sophisticated tool box
- You actually don’t really need the sophisticated tool box
Think about these arguments. Yes, the solution might not be perfect, or even ugly…
But we’re engineers and as such, we’re not here to debate perfection. We’re here to deliver value to our customers and if we can get the job done with the hammer, why not just forget our vanity and simply use that hammer to get the job done.
In a perfect world, queues are transactional and guarantee (to the extent allowed by underlying theory) atomic message delivery or failure – something that we’ve been taking for granted with JMS forever.
At GeeCON Krakow, I had a very interesting discussion with Konrad Malawski regarding his talk about Akka Persistence. If we remove all the hype and buzzwords (e.g. reactive programming, etc.) from Akka, we can see that Akka is just a proprietary alternative to JMS, which looks more modern but is lacking tons of features that we’re used to having in JMS (e.g. like transactional queue persistence).
One of the interesting Aspects of that discussion with Konrad Malawski is the fact that a 100% message delivery guarantee is a myth (details here). Which leads to the conclusion:
Messaging is really hard
It is, indeed! So if you really think you need to embed a sophisticated MQ system, beware of the fact that you will have to learn how it really works and how to correctly operate it.
If you’re using RDBMS-backed queues, you can get rid of this additional transactional complexity, because your queue operations participate in the transactions that you already have with your database. You get ACID for free!
No additional operations efforts
What developers very often underestimate (we can’t say this enough) are the costs incurring to your operations team when you add new external systems to yours.
Having just one simple RDBMS (and your own application) is a very very lean and simple architecture. Having an RDBMS, an MQ, and your application is already more complex.
There are a lot of excellent DBA out there who know what they’re doing when operating productive databases. Finding excellent “MQA” is much harder.
If you’re using Oracle: Use Oracle AQ
Queues in AQ are essentially just tables that contain a serialised version of your message type. If you’re using jOOQ, we’ve blogged about how to integrate Oracle AQ with jOOQ, recently.
RDBMS-centric applications can be much easier
We’ve blogged about that before as well: Why Your Boring Data Will Outlast Your Sexy New Technology.
Assuming that the above really applies to you (it may, of course, not apply): If your RDBMS is in the middle of your system, then running queues in your RDBMS to communicate between system components is quite an obvious choice, isn’t it? All system parts are already connected to the database. Why not keep it that way?
The arguments listed here are all pretty obvious and pragmatic. At some point, they no longer hold true, as your messaging demands are really big enough to justify the integration with a sophisticated MQ system.
But many people have strong opinions about the “hammer / nail” argument. Those opinions may be correct but premature. Very often in software engineering, it is entirely acceptable and sufficient to work with just one tool. The hammer of software: The RDBMS.