Stop Writing Dual-Write Microservices: The Outbox Pattern You Should Be Implementing
Here is a classic anti-pattern found in backend services:
TypeScript
// The dangerous "Dual-Write"
async function createOrder(orderData) {
const order = await db.orders.insert(orderData); // Step 1: DB write succeeds
await eventBus.publish("OrderCreated", order); // Step 2: Network partition / Crash happens here!
return order;
}
If the application crashes, network drops, or the broker rejects the event after Step 1, your database committed state that downstream systems will never know about.
Wrapping both in a distributed transaction (2PC) hurts latency and throughput. Swapping the order—publishing the event first—is worse, because a database write failure leaves phantom events in your event stream.
The Fix: The Transactional Outbox Pattern
Instead of calling your message broker over the network in your application request lifecycle, leverage the ACID guarantees of your primary database:
Atomic Dual-Write in a Single Engine: Create an outbox table in the same database schema as your domain tables. When mutating data, insert your business entity and write the corresponding integration event into the outbox table within the same local database transaction:
SQL
BEGIN TRANSACTION;
INSERT INTO orders (id, customer_id, total) VALUES ('ord_101', 'cust_42', 150.00);
INSERT INTO outbox (id, aggregate_type, payload, status)
VALUES ('evt_201', 'Order', '{"id":"ord_101","total":150.00}', 'PENDING');
COMMIT;
Decoupled Asynchronous Relay: A separate asynchronous worker reads events from the outbox table and publishes them to the broker. You can implement this via:
Polling Publisher: A scheduled query with SELECT ... FOR UPDATE SKIP LOCKED for low-to-medium scale.
Change Data Capture (CDC): Tools like Debezium reading the database write-ahead log (WAL) directly for ultra-low latency and zero database read overhead.
Guaranteed At-Least-Once Delivery: Because your database guarantees the transaction either commits both the entity and the outbox event or rolls back both, you eliminate silent data corruption at the root.
Discussion Question
When decoupling services, do you rely on Change Data Capture (CDC) against the WAL, polling-based outbox processors, or idempotent consumer retries to handle split-brain events?
CTA
Level up your backend architectures and write production-grade code. Join thousands of backend engineers, system designers, and software craftspeople in Developers & Coding.
Here is a classic anti-pattern found in backend services:
TypeScript
// The dangerous "Dual-Write"
async function createOrder(orderData) {
const order = await db.orders.insert(orderData); // Step 1: DB write succeeds
await eventBus.publish("OrderCreated", order); // Step 2: Network partition / Crash happens here!
return order;
}
If the application crashes, network drops, or the broker rejects the event after Step 1, your database committed state that downstream systems will never know about.
Wrapping both in a distributed transaction (2PC) hurts latency and throughput. Swapping the order—publishing the event first—is worse, because a database write failure leaves phantom events in your event stream.
The Fix: The Transactional Outbox Pattern
Instead of calling your message broker over the network in your application request lifecycle, leverage the ACID guarantees of your primary database:
Atomic Dual-Write in a Single Engine: Create an outbox table in the same database schema as your domain tables. When mutating data, insert your business entity and write the corresponding integration event into the outbox table within the same local database transaction:
SQL
BEGIN TRANSACTION;
INSERT INTO orders (id, customer_id, total) VALUES ('ord_101', 'cust_42', 150.00);
INSERT INTO outbox (id, aggregate_type, payload, status)
VALUES ('evt_201', 'Order', '{"id":"ord_101","total":150.00}', 'PENDING');
COMMIT;
Decoupled Asynchronous Relay: A separate asynchronous worker reads events from the outbox table and publishes them to the broker. You can implement this via:
Polling Publisher: A scheduled query with SELECT ... FOR UPDATE SKIP LOCKED for low-to-medium scale.
Change Data Capture (CDC): Tools like Debezium reading the database write-ahead log (WAL) directly for ultra-low latency and zero database read overhead.
Guaranteed At-Least-Once Delivery: Because your database guarantees the transaction either commits both the entity and the outbox event or rolls back both, you eliminate silent data corruption at the root.
Discussion Question
When decoupling services, do you rely on Change Data Capture (CDC) against the WAL, polling-based outbox processors, or idempotent consumer retries to handle split-brain events?
CTA
Level up your backend architectures and write production-grade code. Join thousands of backend engineers, system designers, and software craftspeople in Developers & Coding.
Stop Writing Dual-Write Microservices: The Outbox Pattern You Should Be Implementing
Here is a classic anti-pattern found in backend services:
TypeScript
// The dangerous "Dual-Write"
async function createOrder(orderData) {
const order = await db.orders.insert(orderData); // Step 1: DB write succeeds
await eventBus.publish("OrderCreated", order); // Step 2: Network partition / Crash happens here!
return order;
}
If the application crashes, network drops, or the broker rejects the event after Step 1, your database committed state that downstream systems will never know about.
Wrapping both in a distributed transaction (2PC) hurts latency and throughput. Swapping the order—publishing the event first—is worse, because a database write failure leaves phantom events in your event stream.
The Fix: The Transactional Outbox Pattern
Instead of calling your message broker over the network in your application request lifecycle, leverage the ACID guarantees of your primary database:
Atomic Dual-Write in a Single Engine: Create an outbox table in the same database schema as your domain tables. When mutating data, insert your business entity and write the corresponding integration event into the outbox table within the same local database transaction:
SQL
BEGIN TRANSACTION;
INSERT INTO orders (id, customer_id, total) VALUES ('ord_101', 'cust_42', 150.00);
INSERT INTO outbox (id, aggregate_type, payload, status)
VALUES ('evt_201', 'Order', '{"id":"ord_101","total":150.00}', 'PENDING');
COMMIT;
Decoupled Asynchronous Relay: A separate asynchronous worker reads events from the outbox table and publishes them to the broker. You can implement this via:
Polling Publisher: A scheduled query with SELECT ... FOR UPDATE SKIP LOCKED for low-to-medium scale.
Change Data Capture (CDC): Tools like Debezium reading the database write-ahead log (WAL) directly for ultra-low latency and zero database read overhead.
Guaranteed At-Least-Once Delivery: Because your database guarantees the transaction either commits both the entity and the outbox event or rolls back both, you eliminate silent data corruption at the root.
Discussion Question
When decoupling services, do you rely on Change Data Capture (CDC) against the WAL, polling-based outbox processors, or idempotent consumer retries to handle split-brain events?
CTA
Level up your backend architectures and write production-grade code. Join thousands of backend engineers, system designers, and software craftspeople in Developers & Coding.
Arabic
French
Spanish
Portuguese
Deutsch
Turkish
Dutch
Italiano
Russian
Romaian
Portuguese (Brazil)
Greek