Core Concept

Distributed Transactions: 2PC vs Saga

Cross-service workflows need 2PC when you must block for atomic commits, or Sagas when you can accept eventual consistency and compensating rollbacks.


1. What It Is

The moment your flow touches two databases or microservices, you need a plan for partial failure. We choose between blocking 2PC coordination and async sagas with compensating actions.

What:

Protocols to manage atomic states spanning multiple database partitions or independent microservices.

Primary purpose:

Avoid data inconsistency when operations succeed in one service but crash in another.

Usually used for:

Multi-step payments, distributed ledger transfers, and order fulfillment pipelines.

2. Core Mental Model

When work spans multiple databases, we choose blocking coordination (2PC) or async steps with compensations (Saga):

πŸ”’ Two-Phase Commit (2PC)

Strict locking. Coordinator prepares all workers first. If all say YES, they commit together. If any say NO, they all roll back.

πŸ”„ Compensating Sagas

Asynchronous chain. Services commit local transactions instantly. On mid-pipeline crash, fire reverse undo actions (refunds, cancellations).

πŸ“¬ Reliable Outbox Sync

Write transaction data and event lines inside the SAME database. A background relay tails updates to guarantee event deliveries.

In the room

2PC sounds safe but blocks under partition and doesn't scale β€” most interview answers should default to Saga for microservices. Be ready to name compensating actions: "If payment succeeds but inventory fails, we refund." Mention the outbox pattern for reliable event delivery.

3. Why It Matters in HLD

Cross-service writes need an explicit consistency story β€” 2PC, saga, or outbox. We frame the choice around three lenses:

Needed When:

Operations cross service boundaries (e.g. charge credit card AND reserve warehouse items in separate DBs).

Avoids:

Dangling partial states (e.g. charging credit card without issuing tickets), database locks, and lost event lines.

Optimizes For:

Data consistency guarantees, transaction availability throughput, lock footprint minimizations, and failure recoveries.

4. Architecture & Data Flow

Walk both patterns as parallel interview paths. 2PC path: coordinator prepares all participants, waits for votes, then commits or aborts β€” strong consistency, blocking under partition. Saga path: each service commits locally and publishes events; compensating transactions undo on failure β€” eventual consistency, no global lock. Outbox path: write business row and outbox row in one local transaction; relay publishes to the bus β€” at-least-once with idempotent consumers.

Loading...

Saga Compensation Model

Sagas execute local database commits instantly, executing rollback compensations backward on error:

Loading...

In the room

Default to saga for long-running flows β€” 2PC over HTTP microservices is a red flag. Mention idempotency keys and compensating actions; interviewers listen for those words.

5. Key Characteristics

We compare 2PC and saga on latency, failure behavior, and operational complexity:

  • Two-Phase Commit (2PC) Properties: Lock-heavy, strongly consistent profile:
Property FactorOperational Behavior
AtomicityStrict: All-or-nothing commit guarantees across nodes.
Latency ProfileHigh: Requires 2 round-trip network hops and synchronous disk fsyncs per node.
Locks & BlocksSevere: Holds local database row locks during prepare phases, blocking other queries.
  • Saga Coordination Strategies: Mapping flow control styles:
StyleExecution MechanicCore AdvantageSevere Drawback
ChoreographyDecentralized: Services listen to events and trigger subsequent actions autonomously.
  • Loose coupling
  • no single coordinator bottleneck.
  • Highly complex dependency chains
  • difficult to debug or trace workflows.
OrchestrationCentralized: A stateful orchestrator orders service actions and schedules compensations.
  • Explicit workflows
  • centralized error, retry, and timeout controls.
Orchestrator represents a single point of failure (requires active cluster failover).

6. Strategic Tradeoffs

Strong cross-service atomicity and availability pull apart β€” we state both sides:

Criterion FactorUse 2PC OptionUse Saga Option
Consistency TargetStrict ACID (Money balances, ledger double-entry)Eventual (E-commerce checkouts, ticket bookings)
Node Scale SpanFew nodes (2-3 services max inside single region)Many nodes (3+ services, cross-region arrays)
Lock ToleranceCan tolerate blocking latency to guarantee safetyHighly-available, lock-free high throughput requirement

7. Failure / Bottleneck Awareness

Coordinator failure, poison messages, and duplicate delivery are saga interview staples β€” we name mitigations:

πŸ›‘ Coordinator Lock Stall (2PC)

Problem: Workers hold row locks in the prepare phase. If the coordinator crashes before commit/abort, locks may never release.

Mitigation: Run coordinators in HA pairs, or prefer Saga orchestration on durable workflow engines (e.g. Temporal) for long-running flows.

🐍 Failed Compensation

Problem: If a forward step fails and the compensating action (refund, cancel) also fails, the workflow stalls in a partial state.

Mitigation: Make compensations idempotent, retry with backoff, and route persistent failures to a DLQ for manual repair.

8. Common HLD Usage

Checkout, booking, and payment flows are where these patterns appear on the whiteboard:

The transactional outbox bridges a local DB commit and an outbound event β€” the row and the outbox entry commit atomically, then a relay publishes to the message bus:

Loading...

For Strangler migration and outbox at the microservices layer, see Microservices Patterns: Strangler, Saga, & Outbox (#30).

9. Decision Signals

Think distributed transactions when a user action spans multiple services that must agree or compensate:

🎯 Think Distributed Transactions When:
  • You are designing heavy checkout gateways, travel/flight bookings, or order fulfillment networks spanning independent microservices databases.
  • You must guarantee transaction safety across geographically isolated datacenter regions (AP Sagas).
  • You are required to orchestrate a long-running, multi-step business workflow involving manual human reviews (Saga Orchestration).

11. Deep Dive (Optional)

The Saga Lack of Isolation Problem

Unlike 2PC which holds absolute transaction row locks, Sagas commit local database operations instantly. This means Sagas completely violate the **Isolation (I)** property of ACID. Two distinct anomalies can occur in production:

  1. Dirty Reads (Intermediate State): A customer queries their bank balance while a Saga money transfer is executing step 1 (Debit account) but before step 2 (Credit beneficiary). The reader sees a decreased balance that might rollback later if step 2 crashes.
  2. Lost Updates (Overwriting writes): Step 1 debits balance. A separate concurrent operation updates the account balance without reading the Saga locks. When the Saga rolls back, its compensation overwrites the concurrent update.

**Production Countermeasures**: Implement application-level safeguards like **Pending Flags** (e.g. mark order status as `PENDING_PAYMENT` to prevent shipping actions) or use **Semantic Locks** (e.g. decrement a user's *available* balance, but preserve their *actual* balance until the Saga fully completes).

When 2PC Is Not Enough: TCC & 3PC (Brief)

TCC (Try-Confirm-Cancel) splits each service call into a reserve phase (Try), a commit (Confirm), and an undo (Cancel) β€” common in payment holds where 2PC row locks are too heavy. 3PC adds a pre-commit phase to reduce coordinator-failure blocking but is rarely implemented in production; interviewers mention it to show you know 2PC's coordinator stall problem. Default recommendation remains: 2PC inside one region with few participants, Saga + outbox for cross-service flows.

πŸ’¬Review

Help Us Improve

How helpful was this walkthrough?

Click a star to rate. We actively use this feedback to refine and update our system design content.

Placeholder
Optional but highly appreciated!

Discussion

Share your thoughts, ask questions, or help others.

Loading comments...