The checkout saga

A checkout in Kinetix takes something from four services. pricing gives up a voucher redemption and flash-sale stock, warehouse reserves units on a shelf and opens a picking task, payment moves money into escrow. Each of those lives in its own service and its own PostgreSQL database, so there is no transaction that can hold all of them and roll them back together.

order solves this with a saga: a sequence of local steps, each one committed by the service that owns it, and for each step a compensation that undoes it. If any step fails, the steps already taken are undone in reverse. This lesson follows the saga through order’s code, step by step.

Play it

Four runs of the same checkout. Watch the saga log at the bottom: every row appears before its call goes out.

Text version of this diagram

Everything answers

A cart of two products from one merchant, with a voucher and one item on a flash sale. Before any of this, order priced the cart with pricing, wrote the order as PENDING_PAYMENT and opened a saga. Every step below is written to the saga log as Attempting before its call goes out.

order (checkout)
```
pricing.RedeemVoucher(HEMAT10, ORD-1)
```
Redeem the voucher. pricing records a redemption row for this order — the ledger entry that makes it possible to give the voucher back later.
Code: CheckoutSagaRunner.cs:149–164
order (checkout)
```
pricing.AllocateFlashSaleStock(FS-12, SKU-A, 1, ORD-1)
```
Claim the flash-sale unit before touching the shelf. It is the scarcer resource; failing on it after reserving stock would mean undoing more.
Code: CheckoutSagaRunner.cs:166–182
order (checkout)
```
warehouse.ReserveStock(SKU-A, 1, ORD-1)
```
Reserve the first product in warehouse, under the row lock from the first lesson.
Code: CheckoutSagaRunner.cs:184–200
order (checkout)
```
warehouse.ReserveStock(SKU-B, 2, ORD-1)
```
One row per product: the second reservation is its own step, so each can be released on its own.
Code: CheckoutSagaRunner.cs:184–200
order (checkout)
```
payment.CreateEscrowHold(ORD-1, total, merchant share, shipping fee)
```
Hold the money in escrow. payment moves it from the customer's wallet into a hold, keyed by the order number so a repeat cannot take it twice.
Code: CheckoutSagaRunner.cs:206–222
order (checkout)
```
warehouse.CreateFulfillmentTask(ORD-1, lines)
```
Only now is warehouse told there is an order to pick. A fulfilment task created earlier could be picked for an order that is about to be unwound.
Code: CheckoutSagaRunner.cs:228–248
order (checkout)
```
201 Created
```
Every hold is real, so the saga is Completed, the order becomes PAID — and only then is the cart emptied.
Code: OrderService.cs:173–179

Outcome. Six calls, six rows Done, one answer to the customer: 201 Created.

Warehouse refuses

The same checkout, but the shelf holds only one unit of SKU-B and the cart wants two.

order (checkout)
```
pricing.RedeemVoucher(HEMAT10, ORD-1)
```
The voucher is redeemed.
Code: CheckoutSagaRunner.cs:149–164
order (checkout)
```
pricing.AllocateFlashSaleStock(FS-12, SKU-A, 1, ORD-1)
```
The flash-sale unit is claimed.
Code: CheckoutSagaRunner.cs:166–182
order (checkout)
```
warehouse.ReserveStock(SKU-A, 1, ORD-1)
```
SKU-A is reserved.
Code: CheckoutSagaRunner.cs:184–200
order (checkout)
```
warehouse.ReserveStock(SKU-B, 2, ORD-1)
```
warehouse refuses SKU-B with INSUFFICIENT_STOCK, inside the same transaction that would have taken the units. Nothing is held for this row, so it is recorded Failed and will not be compensated.
Code: CheckoutSagaRunner.cs:184–200
order (checkout)
```
warehouse.ReleaseStock(SKU-A, ORD-1)
```
Compensation starts from the newest row and works backwards. SKU-A goes back to the shelf first.
Code: CheckoutSagaRunner.cs:259–267
order (checkout)
```
pricing.ReleaseFlashSaleAllocation(FS-12, SKU-A, 1, ORD-1)
```
Then the flash-sale unit goes back to the sale.
Code: CheckoutSagaRunner.cs:447–459
order (checkout)
```
pricing.ReleaseVoucherRedemption(HEMAT10, ORD-1)
```
Then the voucher redemption is released, so the customer can use the voucher again.
Code: CheckoutSagaRunner.cs:447–459
order (checkout)
```
409 CHECKOUT_ROLLED_BACK
```
Everything is given back: the saga is Compensated and the order is kept as CANCELLED, not deleted. The customer gets 409 CHECKOUT_ROLLED_BACK with the order number and warehouse's reason.
Code: OrderController.cs:90–95

Outcome. Nothing is held anywhere, and the order explains itself: a cancelled checkout with a reason, not one that silently never happened.

Payment does not answer

The voucher and the stock are held, then the call to payment runs out of time. order cannot tell whether the hold was created before the connection gave up.

order (checkout)
```
pricing.RedeemVoucher(HEMAT10, ORD-1)
```
The voucher is redeemed.
Code: CheckoutSagaRunner.cs:149–164
order (checkout)
```
warehouse.ReserveStock(SKU-A, 1, ORD-1)
```
SKU-A is reserved.
Code: CheckoutSagaRunner.cs:184–200
order (checkout)
```
payment.CreateEscrowHold(ORD-1, total, merchant share, shipping fee)
```
CreateEscrowHold is written as Attempting, sent, and never answered. The row stays Attempting: order does not know whether money moved. The checkout gives up — "a service this checkout depends on did not answer" — and starts compensating.
Code: CheckoutSagaRunner.cs:62–67
order (checkout)
```
payment.RefundEscrow(ORD-1, reason)
```
An Attempting row is compensated exactly like a Done one: undoing too much is safe, undoing too little strands money. The refund goes out under the same order-keyed idempotency key, and had its answer been unclear, order would read payment's ledger with GetEscrowStatus before deciding.
Code: CheckoutSagaRunner.cs:485–518
order (checkout)
```
warehouse.ReleaseStock(SKU-A, ORD-1)
```
SKU-A goes back to the shelf.
Code: CheckoutSagaRunner.cs:447–459
order (checkout)
```
pricing.ReleaseVoucherRedemption(HEMAT10, ORD-1)
```
The voucher redemption is released.
Code: CheckoutSagaRunner.cs:447–459
order (checkout)
```
409 CHECKOUT_ROLLED_BACK
```
The saga is Compensated and the customer gets 409 CHECKOUT_ROLLED_BACK. Whether or not the first hold had landed, nothing is held now.
Code: OrderService.cs:163–171

Outcome. A call with no answer is treated as possibly done. pricing and warehouse answer a release for something never taken as a success; for money, order goes further and confirms against payment's ledger before closing the row.

A release fails

warehouse refuses SKU-B again, but this time pricing is down while the saga unwinds.

order (checkout)
```
pricing.RedeemVoucher(HEMAT10, ORD-1)
```
The voucher is redeemed.
Code: CheckoutSagaRunner.cs:149–164
order (checkout)
```
warehouse.ReserveStock(SKU-A, 1, ORD-1)
```
SKU-A is reserved.
Code: CheckoutSagaRunner.cs:184–200
order (checkout)
```
warehouse.ReserveStock(SKU-B, 2, ORD-1)
```
warehouse refuses SKU-B: Failed, nothing held.
Code: CheckoutSagaRunner.cs:184–200
order (checkout)
```
warehouse.ReleaseStock(SKU-A, ORD-1)
```
Compensation gives SKU-A back.
Code: CheckoutSagaRunner.cs:447–459
order (checkout)
```
pricing.ReleaseVoucherRedemption(HEMAT10, ORD-1)
```
Releasing the voucher fails with UNAVAILABLE. A transient error neither stops the round nor counts as success: the voucher row stays Done, still held. The saga ends Stuck with its next round due in 15–30 s — and the customer already has their 409.
Code: CheckoutSagaRunner.cs:353–363
StuckSagaSweeper
```
pricing.ReleaseVoucherRedemption(HEMAT10, ORD-1)
```
StuckSagaSweeper runs every minute and takes over due sagas under a fresh lease. The retry re-sends only what is still held: the voucher, not SKU-A.
Code: StuckSagaSweeper.cs:75–80
StuckSagaSweeper
```
Compensated
```
The voucher comes back and the saga is Compensated. Had pricing stayed down, the delay would keep doubling up to 30 minutes, and after the sixth round the saga is Abandoned and logged GIVING UP for a person to settle.
Code: CompensationPolicy.cs:39–45

Outcome. A partial unwind is never reported as a complete one. Stuck means still holding something and retrying; Abandoned means a person must look.

Before the saga starts

CheckoutAsync does everything it can before anything is held:

A repeated request with the same Idempotency-Key gets the order it already created, not a second one.
The cart’s products come from catalog, the shipping quote from matching and pricing, and the price from pricing’s CalculatePrice. If pricing cannot answer, checkout stops here.
If pricing’s numbers cannot be charged as they stand — a shipping fee that is not a discount of its own quote, an amount escrow cannot hold — the checkout is refused before an order row exists. Refusing is cheapest while there is nothing to give back.

Only then is the order written, as PENDING_PAYMENT, and the saga started. The order row comes first on purpose: if the saga fails, the order stays as CANCELLED with a reason, so a failed checkout explains itself instead of looking like one that never happened.

The forward pass

RunForwardAsync takes five kinds of step, always in this order:

Step	Service	Why it sits here
`RedeemVoucher`	pricing	Cheap to take and to give back; refuses early when the voucher is spent
`AllocateFlashSaleStock`	pricing	The scarcest thing in the cart — claimed before the shelf, so running out costs the least to undo
`ReserveStock`, one per product	warehouse	One row per product, so each can be released on its own
`CreateEscrowHold`	payment	Money is taken only once the goods are known to exist
`CreateFulfillmentOrder`	warehouse	Last: warehouse is told to pick only an order whose every hold is real

Write the step before the call

BeginStep saves the row as Attempting and commits it before the call goes out; Settle moves it to Done or Failed after the answer. The order matters. Recorded after the call, a process that dies between the two leaves stock or money held with nothing in the database pointing at it. Recorded before, the worst case is an Attempting row for something that never happened — harmless, because undoing it finds nothing to undo. A test asserts the row exists before the call.

Failed is not the same as Attempting

Failed means the service explicitly refused. warehouse refuses inside the same transaction that would have taken the units, pricing without touching the counter. Nothing is held, so the row is not compensated.
Attempting means no answer arrived: the call threw or timed out. order cannot know whether the work happened, so the row is compensated.

Money is the one exception that order double-checks: even a refused escrow hold is checked against payment’s ledger with GetEscrowStatus before order accepts that nothing was taken.

Undoing it

Compensation selects every row still Done or Attempting, newest first, and asks the owning service to give it back: ReleaseVoucherRedemption, ReleaseFlashSaleAllocation, ReleaseStock, RefundEscrow, CancelFulfillmentTask. One release that fails does not stop the others. The round then ends in one of three states:

State	Meaning	What happens next
`Compensated`	Every hold was given back	Nothing; the order is `CANCELLED`
`Stuck`	Something is still held after a transient error	Another round, after a growing delay
`Abandoned`	A permanent error, or the sixth round failed	A `GIVING UP` log line for a person to settle

The delay is 30 seconds, doubling each round, capped at 30 minutes, with jitter, for at most six rounds. Errors that a retry cannot fix — FailedPrecondition, InvalidArgument, NotFound, PermissionDenied, Unauthenticated — abandon at once instead of spending the budget. A partial unwind is never reported as a complete one: a failed release leaves the saga Stuck, and a retry re-sends only what is still held.

Who runs the next round

StuckSagaSweeper wakes every minute and selects sagas that are Stuck and due, still Compensating, or still Running five minutes after anyone last touched them — a checkout whose process died part-way.

Two workers must never unwind the same saga at once, so every saga is driven under a lease: an owner and an expiry written on the saga’s row, which every later write checks. A worker whose lease has expired or changed hands finds that its writes match nothing and stops. The checkout holds the lease while it runs, a sweeper cannot take it from a live checkout, and a worker that has lost its lease cannot write Completed over an unwind.

Why compensation is possible at all

A counter cannot be compensated. used_count - 1 cannot tell whether this order took one, so a retried release hands back quota nobody held, and a retried redeem takes a second helping. Every participant therefore keeps a ledger row per order, and the counter is a cache of it: pricing has voucher_redemptions and flash_sale_allocations, each unique on the order number, and warehouse has the stock_reservations from the first lesson. The unique constraint is the idempotency: redeeming twice finds the row and answers “already redeemed”, releasing something never taken answers “already released” — and both count as success, which is exactly what a saga unwinding after an early failure needs.

What the customer sees

A successful checkout answers 201 Created with an order that is PAID, and only then is the cart emptied. A refused one answers 409 CHECKOUT_ROLLED_BACK with the order number and the reason from the service that refused — an answer, not a fault. If the unwind is still Stuck, the customer already has that answer; the rest happens in the background.

Takeaways

Without a shared transaction, every step needs an owner that commits it and a compensation that undoes it, and every compensation must be idempotent.
Record intent before acting. An Attempting row you cannot explain is cheaper than a hold nothing remembers.
Treat “no answer” as “maybe done” and compensate it; treat an explicit refusal as “nothing held”.
Never report a partial unwind as complete. Retry the transient with backoff, stop on the permanent, and leave a person a clear line to find.