Saga Pattern for Cross-System Workflows

The moment you split a monolith into services, the database transaction goes with it. You can't wrap an order service, a payments service, and a shipping service in a single BEGIN`/`COMMIT` — they don't share a connection, let alone a database. The saga pattern is the answer that has actually held up in production: instead of one atomic transaction, you run a sequence of local transactions, each in its own service, and if one of them fails partway through, you undo the ones that already succeeded by running compensating actions in reverse.

Why two-phase commit lost

The textbook alternative is two-phase commit: a coordinator asks every participant to prepare, waits for unanimous yes, then tells everyone to commit. It gives you real atomicity, and it's a reasonable choice inside a single database cluster. Across independently deployed services it falls apart. A participant that crashes mid-prepare holds a lock that nobody releases until it comes back. Every service in the transaction has to support the same protocol and stay reachable for the duration, which means your availability is the product of everyone's availability, not the best of them. Sagas trade that guaranteed atomicity for something weaker but workable: eventual consistency, achieved by never holding a cross-service lock at all. Each step commits for real, immediately, in its own service. Nothing is left half-open waiting on a coordinator.

Choreography vs orchestration

There are two ways to sequence the steps, and the choice matters more than most teams initially assume. In choreography, there's no central coordinator — each service publishes an event when it finishes its local transaction, and the next service in line reacts to that event and does its own work. The order service publishes OrderCreated, the payment service listens for it, charges the card, and publishes PaymentCaptured, the inventory service listens for that and reserves stock. It's simple to start and there's no single point of failure, but the overall flow only exists implicitly, scattered across every service's event handlers. Once you have five or six steps, tracing "what actually happened to order 4021" means grepping logs across five services and reconstructing the event chain by hand.

In orchestration, a dedicated orchestrator owns the sequence explicitly: it calls the order service, waits for success, calls the payment service, waits, calls inventory, and so on — and on any failure, it's also the thing that walks backward and calls the compensating action for every step that already succeeded. The trade-off is a new component with its own availability and deployment story, and it can turn into a god-object if you let every business rule leak into it. But for anything with more than three or four steps, or any workflow product owners need to see a state diagram for, I reach for orchestration by default. Debugging "what happened to order 4021" becomes one query against the orchestrator's state, not an archaeology dig.

An orchestrator, sequenced and compensated

Here's the shape I actually reach for — an explicit list of steps, each with a forward action and a compensating action, run in order with automatic rollback on failure:

type SagaStep<T> = {
  name: string;
  action: (ctx: T) => Promise<void>;
  compensate: (ctx: T) => Promise<void>;
};

async function runSaga<T>(steps: SagaStep<T>[], ctx: T): Promise<void> {
  const completed: SagaStep<T>[] = [];

  for (const step of steps) {
    try {
      await step.action(ctx);
      completed.push(step);
    } catch (err) {
      // Walk backward through every step that already succeeded
      for (const done of completed.reverse()) {
        try {
          await done.compensate(ctx);
        } catch (compErr) {
          // A failed compensation is the one case that needs a human —
          // log it to a dead-letter queue, don't silently swallow it.
          console.error(`compensation failed for ${done.name}`, compErr);
        }
      }
      throw new Error(`saga failed at ${step.name}: ${err.message}`);
    }
  }
}

const checkoutSaga: SagaStep<{ orderId: string; cardToken: string }>[] = [
  {
    name: 'reserve-inventory',
    action: (ctx) => inventory.reserve(ctx.orderId),
    compensate: (ctx) => inventory.release(ctx.orderId),
  },
  {
    name: 'charge-payment',
    action: (ctx) => payments.charge(ctx.cardToken, ctx.orderId),
    compensate: (ctx) => payments.refund(ctx.orderId),
  },
  {
    name: 'schedule-shipment',
    action: (ctx) => shipping.schedule(ctx.orderId),
    compensate: (ctx) => shipping.cancel(ctx.orderId),
  },
];

Every step in that list needs a compensate function before it's allowed in, which is the useful discipline the pattern forces on you: if you can't say how to undo "charge the payment," you've found a design problem before it's in production, not after.

Idempotency is not optional

Sagas run over unreliable networks, which means every action and every compensation will occasionally be retried — the orchestrator crashes after calling payments.charge but before recording that it succeeded, restarts, and calls it again. If charge isn't idempotent, you double-bill the customer. The fix is the same one you'd use for any at-least-once delivery system: every action takes an idempotency key derived from the saga instance and step name, and the receiving service stores it and short-circuits a repeat. This isn't a saga-specific concern, but it's the detail that turns a saga from "works in the demo" to "survives a real network partition," and it's the one people skip because it doesn't show up until the second or third production incident.

A choreography event, for comparison

The equivalent choreography step is just an event contract — no central sequence, just a well-defined message each service knows how to react to and how to compensate for if a later event signals failure:

type OrderEvent =
  | { type: 'OrderCreated'; orderId: string; cardToken: string }
  | { type: 'InventoryReserved'; orderId: string }
  | { type: 'InventoryReservationFailed'; orderId: string; reason: string }
  | { type: 'PaymentCaptured'; orderId: string }
  | { type: 'PaymentFailed'; orderId: string; reason: string };

// Payment service reacts to inventory succeeding, and to its own failure
// by publishing a compensating event the inventory service listens for.
async function onInventoryReserved(evt: OrderEvent) {
  if (evt.type !== 'InventoryReserved') return;
  try {
    await payments.charge(evt.orderId);
    publish({ type: 'PaymentCaptured', orderId: evt.orderId });
  } catch (err) {
    publish({ type: 'PaymentFailed', orderId: evt.orderId, reason: err.message });
  }
}

Notice what's missing: nothing in this file knows the full checkout flow. The inventory service has to separately subscribe to PaymentFailed and know to release its reservation. That knowledge is correct, but it lives in a different file, in a different service, deployed by a different team. That's the real cost of choreography — not that it's wrong, but that the sequence is a fact about the system that exists nowhere you can read it in one place.

Wrapping up

Pick choreography for two or three services with an obviously linear flow and no need for a central view of progress. Pick orchestration once you have more steps than that, or once someone asks "what state is order 4021 in" often enough that you need a real answer. Either way, the pattern only works if you design the compensating action before you write the forward one, and if every action is idempotent from day one — retries are not the exception case for a saga, they're the normal case.