The Outbox Pattern: Reliable Events Without 2PC
A user places an order. Your service opens a transaction, writes the row to Postgres, commits, and then calls broker.publish("order.created"). The commit succeeds. Then the broker call throws, because the network blipped or the process got OOM-killed in the half-second between the two operations. Your database now says the order exists. Every downstream service, billing, inventory, notifications, believes it never happened.
Flip the order of operations and you get the mirror image: publish first, then the DB transaction rolls back. Now the event is out in the world describing an order that does not exist. This is the dual-write problem, and no amount of careful ordering fixes it. You need both writes to be atomic, but they live in two different systems. This post shows you how to make them atomic without the distributed-transaction machinery everyone tells you to avoid.
Why not just use 2PC?
The textbook answer to “make two systems commit atomically” is a distributed transaction, usually two-phase commit (2PC) coordinated over XA. In principle it works. In practice you rarely want it.
2PC needs a transaction coordinator that all participants trust, it holds locks across the whole prepare-commit window (so a slow or dead participant stalls everyone), and it fails badly in the “in-doubt” state where the coordinator dies after prepare but before commit. Worse, most modern message brokers, Kafka and the typical cloud queue, do not offer a usable XA resource manager at all. So even if you wanted 2PC, the broker half of your dual write usually cannot participate. You end up needing a different guarantee: not “both commit together” but “if the DB commits, the message eventually gets published, exactly the ones that were committed.”
The transactional outbox
The insight is to stop treating the message as a separate write. Instead of publishing to the broker inside your business transaction, you insert a row into an outbox table in the same database, in the same transaction as the state change. One transaction, one database, fully atomic. Either the order row and its outbox row both commit, or neither does. There is no window where one exists without the other.
A separate process, the message relay, reads unpublished rows from the outbox and pushes them to the broker afterwards. The relay is the only component talking to the broker, and it runs asynchronously, so a broker outage never rolls back your business write. It just delays delivery.
sequenceDiagram
participant App as Order service
participant DB as Postgres (orders + outbox)
participant Relay as Relay / poller
participant Broker as Message broker
App->>DB: BEGIN
App->>DB: INSERT order
App->>DB: INSERT outbox row
App->>DB: COMMIT (atomic)
loop every N ms
Relay->>DB: SELECT unsent FOR UPDATE SKIP LOCKED
Relay->>Broker: publish(event)
Relay->>DB: mark sent / delete
end
This is a core data pattern in microservices, and it underpins most reliable integration patterns between services.
The outbox table
Keep the schema close to what a change-data-capture tool expects, so you can switch relays later by configuring the tool rather than migrating the table. Debezium’s outbox event router, for example, defaults to columns named aggregatetype, aggregateid, type, and payload, and every one of those is remappable via route.by.field and the table.field.event.* options if your names differ.
CREATE TABLE outbox (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
aggregate_type text NOT NULL, -- e.g. 'order'
aggregate_id text NOT NULL, -- e.g. the order id
event_type text NOT NULL, -- e.g. 'order.created'
payload jsonb NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
published_at timestamptz -- NULL until the relay sends it
);
-- The poller only ever wants the unsent rows, oldest first.
CREATE INDEX outbox_unpublished_idx
ON outbox (created_at)
WHERE published_at IS NULL;The partial index matters: the outbox grows without bound if you never prune it, but the index stays tiny because it only covers rows still waiting to go out.
Writing the event and the state change together
The whole point is that these two inserts share one transaction. In Node with pg:
async function createOrder(pool, order) {
const client = await pool.connect();
try {
await client.query("BEGIN");
await client.query(
"INSERT INTO orders (id, customer_id, total_cents, status) VALUES ($1,$2,$3,'created')",
[order.id, order.customerId, order.totalCents]
);
await client.query(
`INSERT INTO outbox (aggregate_type, aggregate_id, event_type, payload)
VALUES ('order', $1, 'order.created', $2)`,
[order.id, JSON.stringify({ orderId: order.id, total: order.totalCents })]
);
await client.query("COMMIT");
} catch (err) {
await client.query("ROLLBACK");
throw err;
} finally {
client.release();
}
}No broker call anywhere in that function. If COMMIT succeeds, the event is durably queued. If anything fails, both inserts vanish together.
The polling relay, with SKIP LOCKED
The simplest relay polls the table on an interval. The trap is running more than one relay instance: two pollers grab the same rows and you double-publish, or they serialize on row locks and throughput collapses. FOR UPDATE SKIP LOCKED solves both, letting each poller claim a disjoint batch without blocking. I go deeper on the mechanics in turning Postgres into a job queue with SKIP LOCKED.
async function relayOnce(pool, broker) {
const client = await pool.connect();
try {
await client.query("BEGIN");
const { rows } = await client.query(
`SELECT id, aggregate_id, event_type, payload
FROM outbox
WHERE published_at IS NULL
ORDER BY created_at
FOR UPDATE SKIP LOCKED
LIMIT 100`
);
for (const row of rows) {
await broker.publish(row.event_type, {
key: row.aggregate_id, // partition key preserves per-aggregate order
id: row.id, // stable dedup key for consumers
data: row.payload,
});
await client.query(
"UPDATE outbox SET published_at = now() WHERE id = $1",
[row.id]
);
}
await client.query("COMMIT");
} catch (err) {
await client.query("ROLLBACK"); // rows stay unpublished, retried next tick
throw err;
} finally {
client.release();
}
}
setInterval(() => relayOnce(pool, broker).catch(console.error), 500);Because the UPDATE and the whole batch commit together, a crash mid-loop simply leaves those rows unpublished and the next tick retries them.
At-least-once, so consumers must be idempotent
Notice the gap: the relay publishes to the broker, then marks the row sent. If it crashes in between, the row is still NULL and gets published again on the next run. That is unavoidable with two systems, and it is exactly why the outbox gives you at-least-once delivery, never exactly-once. Duplicates will happen.
The fix lives on the consumer side, not the producer. Every consumer must be idempotent: dedupe on the stable id you carried in the message, or make the handler naturally repeatable. This is the same discipline that stops double charges with idempotency keys, and the same guarantee you build webhook receivers around in webhook delivery, retries, and signature verification.
Ordering
ORDER BY created_at in the poller plus a broker partition key (aggregate_id) gives you per-aggregate ordering: all events for one order arrive in the sequence they were written. You do not get, and rarely need, a global total order across every aggregate. If two orders’ events interleave, no consumer cares. Keep the ordering guarantee scoped to the aggregate and your throughput stays high.
When to reach for CDC instead
The polling relay is honest, boring, and correct, but it queries the DB on a timer. At high volume, or when you want the outbox invisible to your application code, use transaction log tailing instead: a tool like Debezium reads the Postgres write-ahead log, turns each committed outbox insert into a Kafka event, and routes it by aggregate_type. No polling, no published_at bookkeeping, lower latency. The trade is operational weight: you now run Kafka Connect and a connector. Start with the poller. Graduate to CDC when the poll interval or the load actually hurts.
Takeaway
You cannot atomically write to your database and your broker, and 2PC is the wrong tool for trying. Write the event to an outbox row inside the same transaction as your state change, then let a relay publish it afterwards. One database, one transaction, zero distributed commits. Accept at-least-once delivery, make your consumers idempotent, scope ordering to the aggregate, and reach for CDC only when polling stops being enough. The dual-write problem does not get solved by careful ordering. It gets solved by refusing to have two writes.