What to do when a later step fails for good after an earlier one succeeded somewhere you can't roll back. One page.
The rule
You can't erase a charge. Write a new fact that cancels it (a refund), deliver that fact the same way you deliver everything else (outbox, key, at-least-once), and believe it only when the provider confirms it.
Words
term
meaning
saga
"a reliably coordinated sequence of local transactions" (microservices.io). If one fails for good, run compensations for the ones already done.
compensation
undoes a step "from a semantic point of view", not by restoring the old state (Garcia-Molina & Salem, 1987). A refund, a cancellation.
pivot
the step with no compensation (the parcel ships, the email goes out). Put it last.
lack of isolation
others can see the in-between state ("charged, refund on its way"). Design the UI for it.
2PC
two-phase commit: all participants lock and wait for a coordinator. External APIs won't take part, so we don't use it.
State lives in its own row: fulfilment.state ∈ shipped · compensating · refunded. orders.status stays paid: the charge did happen.
Three rules
rule
how
naive version, measured (npm run l7, seed 1)
1. Compensate only on a definite no
lost or late event → ask Shipping (sweepStuck). null or a failed call → wait.
refund-on-silence: 15 of 200 shipped AND refunded
2. Outbox + key
rejection → inbox + compensating + refund.requested outbox row in one tx. Relay → issueRefund with key refund:${orderId}.
refund-inline: 3 never refunded (crash), 4 "already refunded" 400s for refunds that worked
3. Verify, ask, dedupe
signature → getCharge now → one tx: inbox, compensating → refunded (conditional), one ledger row
ignore (L6 flow): 21 charged, never shipped, never refunded
The four functions (src/compensation.ts)
function
does
returns
onShippingEvent(deps, ev)
one tx: inbox dedupe · order not paid → ignore · shipped → shipped · rejected → compensating + outbox row. Never calls PayCo. Then crash point after-event-commit.
applied · duplicate · ignored
issueRefund(deps, msg)
PayCo refund through net (timeout + retry), key from the order · crash point after-refund-call · store refund_id if none. State stays compensating (refund is pending).
the refund
onWebhook(deps, ev)
bad signature → ValidationError · getCharge through net; failure → throw, record nothing · one tx: inbox, then only if that refund succeeded: compensating → refunded + one ledger row
applied · duplicate · ignored
sweepStuck(deps)
paid orders without a fulfilment row → statusOf through net · event → onShippingEvent · null / failure → leave it
{ resolved, waiting }
Webhooks (Stripe's contract, which PayCo copies)
fact
so you…
forgeable: "Always verify that webhook events originate from Stripe before acting on them"
check the signature first (HMAC, constant-time compare)
"Stripe doesn't guarantee the delivery of events in the order that they're generated"
don't trust the payload's status: fetch the object now
"might occasionally receive the same event more than once"
"logging the event IDs you've processed" = an inbox
retried "for up to three days" until you answer 2xx
on a failed lookup, throw and record nothing
two different events can describe the same fact
keep a state guard (WHERE state = 'compensating') beside the inbox
Refunds (Stripe)
"you can't refund a total greater than the original charge amount". Without a key, a retried full refund gets a 400 for a refund that already worked.
Refunds can be pending, then succeeded, and rarely failed (refund.failed). The lab has no failed refunds; production needs a path for them (alert, and another way to pay the customer back).
A refund issued soon after the charge may show up as a reversal: the charge disappears from the statement.
Invariants (lib/saga.ts, per order, against ground truth)
invariant
protected by
shipped once or refunded once; never both; never neither
rule 1 (no refund on silence) · rule 2 (outbox delivers the request)
npm run test:07 # 14 tests
npm run l7 # Part A (naive shops) + Part B (your saga, 50 worlds)
SEEDS=200 npm run l7 # → results/07-compensation.json (evidence: src/, no SEED, ≥ 50 seeds)
REJECT=0.3 npm run l7 # more rejections → results/07-compensation.reject0.3.json
IMPL=mutant SEEDS=50 npm run l7 # → results/07-compensation.mutant.json
Still not handled
A refund that fails (closed card): the saga needs another compensation path, or a human.
Your webhook endpoint down for more than three days: Stripe stops retrying. Poll by stored id as a backstop.
Partial rejections (one item of three out of stock): partial refunds, each with its own key (refund:o7:item-2).
Orchestration: here each event triggers the next step (choreography). Longer sagas often use one coordinator with a durable log.
Go deeper
Applying the Saga Pattern▶ Video · 34 min · Caitie McCaffrey, GOTO 2015The paper's idea carried to services that can't share a transaction.
Pattern: Saga📄 Article · 10 min · Chris RichardsonChoreography vs orchestration; drawbacks.
Sagas📄 Paper · Garcia-Molina & Salem, 1987Semantic undo; the airline-seat example.