rabbitmq · pulsar
Messaging & distributed systems
Queues back up at peak times, messages get lost or processed twice, and one busy customer slows everyone else down.
The problem
Messaging is easy to start and hard to run at volume. A design that worked for thousands of messages a day falls over at millions: queues back up, consumers fall behind, retries create duplicates, and a single busy tenant starves everyone else. Because the problems show up under load, they often show up in production first.
How we approach it
- Design for the volume you'll have. We model the flows, peak rates and ordering rules first, then choose the broker and topology that fit, whether that's RabbitMQ, Apache Pulsar or Azure Service Bus.
- Keep tenants apart. Separate topics, namespaces or queues, with limits per tenant, so one customer's spike doesn't become everyone's outage.
- Process safely at high concurrency. Multithreaded and asynchronous consumers that acknowledge only after work is done, handle retries without duplicates, and shut down without losing in-flight messages.
- Watch the queues. Backlog size, age of the oldest message, consumer count and throughput on one dashboard, with alerts on what actually signals trouble.
- Migrate without a big-bang cutover. Moving between brokers happens one flow at a time, with the new path running in shadow mode before it takes over.
We have hands-on experience building high-throughput, multitenant messaging systems in .NET and moving them between brokers.
What you get
- A messaging architecture with flows, topics and failure handling documented
- Consumers and producers delivered as pull requests in your repositories
- Queue-health dashboards and alerts
- Load tests that show the system holding at your expected peak
How we scope it
Every engagement starts with a short design review of your current flows. We agree on scope, deliverables and price before any delivery work begins.
questions
Messaging & distributed systems: common questions
RabbitMQ, Apache Pulsar or Azure Service Bus: which should we use?
RabbitMQ is a strong default, Azure Service Bus fits well if you're on Azure and want it managed, and Apache Pulsar earns its complexity at very high volumes or when you need long retention and replay. See when Pulsar makes sense.
Our queues keep backing up. What can you do?
We measure where messages wait and why, then fix the bottleneck: consumer concurrency, batching, poison messages, or the downstream system. Then we add monitoring so you see it coming next time.
How do you avoid losing or duplicating messages?
With acknowledgements, retries with backoff, dead-letter queues and idempotent consumers, so a message processed twice does no harm.