Event-Driven Architecture and Message Queues for Business Applications: Fast, Spike-Resistant, and Easy to Extend

When a customer taps the pay button in an online store, many things have to happen: the order is recorded, stock is reduced, an invoice is created, a confirmation email is sent, the warehouse is notified, loyalty points are added, and the data enters reports. In a simply designed application, all those steps are done in sequence in a single process. While orders are few, it seems fine. The moment one step slows down, for example the email server stalls, the entire order is held up, and the customer stares at a spinning screen wondering whether the payment went through.

Event-driven architecture and message queues are a way to design applications so those steps do not hold each other hostage. This article explains the concepts without excess jargon, when the approach is worth using in business applications, common patterns, risks to manage, and how to start gradually without rewriting the whole system.

Summary

• A message queue separates work that must finish now from work that can follow, so users do not wait for secondary steps
• Event-driven architecture lets modules notify each other through events rather than calling each other directly, so they are more loosely coupled
• Main benefits: resilience to load spikes, isolated failures, and ease of adding new functions
• The price: complexity, data whose consistency is delayed, and a need for better monitoring
• Start with one clearly problematic flow, such as notifications or synchronization to another system, rather than overhauling everything

The problem with flows done in sequence

In a synchronous design, a single user request triggers a chain of work that must all finish before the answer is sent back. The pattern is simple but creates some familiar problems.

• Slow in the user's eyes: the waiting time is the sum of all steps, including steps that do not really need to be waited for, such as sending email
• Fragile to failure: if a third party service is down, the whole process fails even though the core work already succeeded
• Hard to absorb spikes: during a campaign or promo, thousands of requests pile up at the weakest point and take down the whole application. This relates to [load testing before a big promo](/blog/load-testing-uji-beban-sebelum-promo-besar-dan-musim-puncak)
• Tight coupling: adding a new function, such as sending data to an analytics system, means changing the main flow's code and risking breaking it
• Unsafe repetition: if a user presses the button twice because it is slow, work may be duplicated

A real example: the checkout process calls the payment service, then the shipping service, then the accounting system, then email. If the accounting system is down for five minutes of maintenance, a customer who already paid gets an error message. Payment happened, but the order appears to have failed, and the support team has to clean it up manually.

Basic concepts: queues, events, and workers

A few terms are enough to understand at the level of ideas.

Message queue

A message queue is an orderly temporary store for work waiting to be processed. The sender puts in a message, for instance "send a confirmation email for order 1042", and immediately continues with its task. Another worker takes that message and handles it when ready. If the worker is busy or briefly down, the message waits safely.

Event

An event is a record of the fact that something has happened, such as "order created", "payment received", or "stock running low". Naming in the past tense emphasizes that an event cannot be disputed; all other parties can do is react to it.

Producers and consumers

A producer is the part of the application that announces an event or puts work in the queue. A consumer is the part that listens and reacts. One event can be heard by many consumers at once: an "order created" event can trigger stock reduction, invoice creation, and notifications without the producer needing to know who they are.

Message broker

The broker is the component that maintains queues and arranges delivery of messages to consumers. Options vary, from managed cloud services to self-installed software. For most mid-sized businesses, a managed service reduces the operational burden compared with running it yourself.

Patterns commonly used in business applications

Background jobs

The simplest and most often useful pattern: work the user does not need to wait for is moved to a queue. Examples are sending emails and notifications, generating large report files, resizing uploaded images, and importing data files. The user immediately gets an "accepted" response while the process runs behind the scenes.

Publish and subscribe

One event is announced, many parties subscribe. This pattern suits interdependent ERP modules. When a sale is recorded, the stock, finance, and CRM modules each react. Adding a new consumer, such as an analytics module, does not change the producer.

Synchronization between systems

Copying data changes from one system to another, such as from an ERP to an online store or marketplace, suits queues very well. A stock change is sent as an event, and if the destination system is unavailable, the event waits and is delivered again. This pattern complements webhooks and real-time integration and is relevant to unifying physical store and marketplace stock.

Spike smoothing

When traffic surges, requests are held in the queue and processed at a rate the system can bear. Users may wait slightly longer for the final result, but the application does not collapse.

Long-running workflows

Business processes that involve several stages and may take time, such as tiered approvals or order fulfillment, can be modeled as a series of events with the status recorded at each step.

Risks and the price to pay

This approach is not a silver bullet. Some consequences need to be understood before deciding.

• Delayed consistency: after an order is created, stock may only drop a few seconds later. The application must be designed so the display still makes sense during that gap
• Duplicate messages: queue systems generally guarantee delivery at least once, so the same message may arrive twice. Consumers must be safe to repeat (idempotent), for example by recording the numbers of messages already processed
• Order is not always guaranteed: an "order cancelled" event may arrive before "order created" if the paths differ. The design needs to handle ordering or use version numbers
• Poison messages: a message that always fails processing can clog the queue. Use retry limits and a dedicated queue for failed messages so a human can inspect them
• Hard to trace: one user action spreads to many consumers, so without correlation identifiers and [good observability](/blog/observabilitas-log-dan-penanganan-error-aplikasi-bisnis), finding the root cause becomes difficult
• Operating cost: one more component to monitor, secure, and back up
• Testing complexity: testing asynchronous flows is harder than calling a single function

For that reason, do not use queues for things that are fine handled directly. If an operation is fast and must be confirmed successful before answering the user, such as validating a card or reducing scarce stock, a synchronous approach is often simpler and safer.

Keeping data correct: idempotency and transactions

Two technical practices determine whether an event-driven design can be trusted.

Idempotency

An idempotent operation produces the same result even when run many times. For example, "mark order 1042 as paid" is safe to repeat, while "add 100,000 to the balance" is not. Design consumers to check whether a message has already been processed, using a unique message number or order number as the key. The same principle applies to notifications from payment providers, as discussed in payment gateway integration.

Reliable event storage

A subtle problem arises when an application saves data to a database and then sends a message to the queue as two separate steps. If the application dies between them, the data is saved but the message is not sent, or the reverse. The outbox pattern solves it: the message is written to a dedicated table in the same database transaction as the data change, and a separate process then forwards it to the queue. That way the data change and the announcement of the event always travel together.

When the approach is worth using

Not every application needs it. Some signs that queues and events will add value:

• There are slow or unreliable steps, such as email, report exports, or calls to external systems, that make users wait
• Seasonal or campaign load spikes make the application stall
• Several systems need to know about the same change, for example ERP, online store, and analytics
• A third party system failure must not fail the core transaction
• You plan to add many integrations in the future and want to avoid tangled code

By contrast, a small application with a few dozen users and simple flows is usually better without extra components. Consider also the relationship with the decision between monolith or microservices: queues can be used inside a tidy monolithic application without having to split it into many services.

Monitoring the queue

An unmonitored queue is a problem waiting to happen. Some indicators worth setting alerts on:

• Queue length: the number of messages waiting. A steady rise means consumers are not keeping up
• Age of the oldest message: how long the longest-waiting message has been there. This is often more meaningful than queue length
• Failure and retry rate: a spike signals trouble in a consumer or the destination system
• Contents of the failed message queue: every message there needs review and a decision on its fate
• Processing time per message: a rising trend can signal performance problems, including in the database as discussed in [slow database and application performance](/blog/performa-database-dan-aplikasi-bisnis-yang-melambat)
• Worker resource usage: determines when to add capacity

Record a correlation identifier on every message so one user action can be traced from the start through all the consumers that reacted.

Steps to start gradually

• Identify one clearly painful flow, such as a checkout slowed by email or stock synchronization that often fails
• Move only the secondary steps to the queue, and leave the core steps synchronous
• Design messages with a unique identifier, a version, and just enough data, not the entire object
• Make consumers idempotent and define a retry policy and a failed message queue
• Put monitoring and alerts in place before launch, not after problems appear
• Test failure scenarios: stop the consumer, send duplicate messages, and simulate a slow destination system
• Launch gradually, for example with [feature flags and gradual rollouts](/blog/feature-flag-dan-rilis-bertahap-untuk-aplikasi-bisnis), then extend to other flows once proven stable

Common mistakes

• Using queues for everything, including things that are simple and must be synchronous
• Not handling duplicate messages so customers receive two emails or are billed twice
• Letting failed messages pile up with nobody checking
• Sending large objects in messages so data quickly becomes stale or inconsistent
• Not including a correlation identifier so tracing a problem takes days
• Ignoring security: messages may contain personal data so access and encryption need to be set up

Frequently asked questions

Does a small business need a message queue?

Often not yet. Start with the background job feature provided by your application framework. A separate broker component only pays off when volume, the number of connected systems, or resilience demands rise.

What is the difference between a queue and a webhook?

A webhook is how one system notifies another through an HTTP call when something happens. A queue is a holding area that keeps messages safe if the receiver is unavailable. The two are often used together: an incoming webhook is first placed in a queue, then processed.

Is this the same as microservices?

No. Queues and events are communication patterns that can be used in monolithic applications as well as microservices. Many businesses gain a lot from queues without splitting their application.

What about the security of data in messages?

Limit personal data in messages, use identifiers to refer to full data, set access rights to the queue, and enable encryption in transit and at rest as needed. Failed messages kept for a long time also need a retention policy.

How long does it take to implement for one flow?

For a simple flow such as notifications, days to a few weeks, including testing and monitoring. Flows involving synchronization between systems or financial data take longer because failure testing is more demanding.

Conclusion

Event-driven architecture and message queues help business applications stay fast in the user's eyes, withstand spikes and disruptions, and grow without dismantling the main flow. The price is added complexity that must be managed with idempotency, failed message handling, and good monitoring. Start small on a clearly problematic flow, measure the result, and expand if it proves useful.

Agsora helps design and build reliable integrations between systems, from ERP and online stores to third party services, including failure handling and monitoring. If your application often stalls when busy or synchronization between systems cannot be relied on, let us discuss the solution.