Microservices • Saga Pattern • CDC

Designing Saga Choreography for Workflow Engines

Netflix Conductor orchestrates 30+ critical workflows inside our Freshworks iPaaS. Pairing saga choreography with Change Data Capture keeps microservices consistent without the latency tax of two-phase commit.

Author: Himanshu Sharma Updated: Aug 2026 12 min read

60%fewer reconcilers

30+orchestrated workflows

hoursto onboard a new saga

The distributed consistency problem

Omnichannel and iPaaS flows touch CRM, billing, messaging, and identity. A ticket create might also provision a bot session, write a knowledge-graph edge, and fan out an SMS. If any hop fails after a commit, you get split-brain state: CRM thinks the ticket exists, messaging does not.

Two-phase commit across these systems is a non-starter. Latency budgets are sub-100 ms at the edge, partners time out, and lock duration across DynamoDB + Postgres + third-party APIs is operationally unsafe.

Treat 2PC as a last resort. For integration platforms, sagas plus an authoritative event log are the default consistency model.

Why saga + CDC

Saga choreography lets each service own its forward action and its compensation. CDC makes those state changes visible to Conductor in near real time so the engine can decide progress, retry, or rollback without polling databases.

Approach	Strength	Failure mode
Orchestration (central commander)	Easy to visualize	Control plane becomes a bottleneck
Choreography (events)	Services stay autonomous	Hard to debug without a contract
Saga + CDC (chosen)	Durable, inspectable, language-agnostic	Requires event taxonomy discipline

Reference architecture

Domain services write to their system of record. DynamoDB streams (or Debezium on Postgres) feed Kafka. Conductor workers consume domain topics and advance the workflow using task-domain routing.

CDC event stream Active service / worker

Services emit CDC events; Conductor owns saga progress, not business writes.

Partition Kafka topics by aggregate id (ticket id, account id) so ordering holds per saga instance. Cross-aggregate fan-out uses separate consumer groups with idempotent handlers.

Event timeline (single saga instance)

A ticket-create saga might emit six events in under two seconds. Conductor advances only when the expected CDC envelope arrives.

Each dot is a durable checkpoint; Conductor never guesses state.

Happy path vs compensation path

Forward progress Compensation branch

Forward path is linear; failure always runs a documented reverse sequence.

01 Intent

API records command id + aggregate key.

02 Forward

Each service commits locally, then emits CDC.

03 Decide

Conductor matches event to next task or compensation.

04 Close

Terminal status is indexed on workflow_summary.

Workflow contract

Every workflow declares a task domain contract so execution routes to language-specific workers (Java WebFlux vs Node.js). Domains also bound throttling, retries, and alerting. Upstream PR #492 added task domain to the summary index so operators can search by domain.

Expose the contract as metadata, not tribal knowledge. UI, APIs, and on-call dashboards should all query the same fields.

Choreography guardrails

Idempotent tasks with deterministic compensation keys.
Time-boxed states with heartbeats so Conductor can retry or compensate.
Tracing parity via OpenTelemetry baggage from CDC to Grafana.
Poison-event quarantine after N retries, never silent drop.
Idempotency tokens on public terminate/start APIs (see also PR #766 for duplicate termination).

Metrics that matter

Median and p95 orchestration time per workflow type.
Compensation rate per task domain (alert if it spikes).
CDC lag from source table to Conductor consumption.
Duplicate terminal notifications (should be ~0 after the terminate guard).

Combining CDC + saga cut reconciliation incidents by 60%, and a new workflow now onboards in hours rather than days.