Design a Multi-Agent Orchestration Platform (LangGraph / Supervisor-Worker)

1. Supervisor-Worker State Graph ⭐

Next we explore each architectural component. Supervisor routing and durable execution form the operational core, while role-scoped tool allowlists prevent privilege escalation by underlying models.

The graph operates as a state machine rather than a DAG batch pipeline. After each worker completes, control returns to the supervisor node, which evaluates the updated shared state and determines the next hop, including cycles between coder and reviewer or routing to the terminal END state.

System: You are the supervisor router for a multi-agent workflow.
You read the shared graph state and decide which specialized worker runs next.

Shared state (authoritative):
  task: string
  artifacts: {type, content_ref, producer_agent}[]
  open_questions: string[]
  budget: {tokens_remaining, usd_remaining, wall_clock_deadline}
  hitl_pending: boolean

Available next nodes:
  - researcher: web search, document RAG, summarize sources
  - coder: draft and edit documents, optional cloud sandbox execution
  - reviewer: policy check, redline, compliance flags
  - human_gate: pause until operator approves (required before external send)
  - END: task complete, emit final artifact

Rules:
  - Never route to coder until researcher has at least one cited source (unless task is code-only).
  - Route to human_gate before any action with external side effects (email, PR merge, API write).
  - If budget.tokens_remaining < 10%: prefer reviewer summary + END over more research.
  - Output JSON only: {"next_node": "...", "reason": "...", "state_patch": {...}}

2. Shared State vs Message Passing

Shared state serves as the architectural default: a single structured JSON document exists per run and is patched after each node execution. Both the supervisor and worker agents inspect the same artifacts and budget meters, comfortably scaling to roughly 256 KB of structured metadata per run.

• Pros: Presents a clean mental model (matching LangGraph StateGraph), provides the supervisor with a global view of task progress, and simplifies human-in-the-loop preview generation.
• Cons: Creates a hot database row requiring frequent transactional updates, necessitating that large artifacts are stored as blob references rather than embedded inline.
• Message passing complement: Appends events to Message Queues for analytics and downstream ingestion, ensuring worker execution never depends on broker availability for correctness.
• Replay recovery: Reconstructs state from the event log via Event Sourcing and CQRS in the event of checkpoint corruption.

3. Tool Router and Specialized Agents

Role-scoped tool allowlists enforce strict least privilege across workers, preventing sensitive data exfiltration or unintended resource allocation.

• Researcher: Restricted to web_search, rag_query (via Document QA Platform backed by Vector Database), and summarize, with no mutation rights on external systems.
• Coder: Authorized for write_artifact and edit_section, with optional access to spawn_sandbox delegating to an Autonomous Cloud Coding Agent environment for code validation.
• Reviewer: Permitted to run policy_check, redline, and append review comments into shared state, with network and sandbox tools completely blocked.
• Supervisor: Limited exclusively to routing decisions without direct tool invocation capabilities. Worker credentials and the tool router remain the enforcement boundary, so the supervisor model cannot grant itself a privileged tool.

4. Durable Execution via Temporal and Checkpointing ⭐

Each graph run maps directly to a dedicated Workflow Orchestration (Temporal) workflow. Node executions run as discrete activities equipped with timeouts, heartbeats, and retry policies. Checkpointing every N steps materializes shared state to ensure rapid worker recovery.

Per-node cost/latency budgets (enforced before dispatch):

Node            Max tokens   Max latency   Model tier (via LLM Gateway)
────────────────────────────────────────────────────────────────────────
supervisor      4K           3s p95        fast router (Haiku / GPT-4o-mini)
researcher      32K          60s p95       Sonnet + tool calls
coder           48K          120s p95      Sonnet, cloud sandbox billed separately
reviewer        16K          30s p95       Sonnet or dedicated compliance model
human_gate      0            24h SLA       no LLM, Temporal timer

Global run budget is waterfall: child nodes cannot exceed run.tokens_remaining.
On breach: supervisor forced-routes to END with partial artifact + budget_exhausted flag.

Checkpoint every N=5 steps (configurable):
  - Persist full shared state to graph_checkpoints at the checkpoint boundary
  - Append GraphCheckpointCreated to graph_events (append-only log)
  - Await the durable checkpoint before dispatching the next node so the recovery point is committed
  - Low N causes write amplification, while high N increases replay cost on failure

• Workflow lifecycle: Coordinates the supervisor loop, activity dispatch, checkpoint persistence, and graph-version pinning across process restarts. Long runs use continue-as-new before workflow history approaches configured limits.
• Idempotent activities: Uses idempotency keys (as detailed in Idempotency and Exactly-Once Effects) on tool executions to avoid duplicate side effects upon retry.
• Signals: Handles external events such as approval, rejection, run cancellation, or runtime operator steering hints merged into the state patch.

5. Human-in-the-Loop Gates

Human-in-the-loop nodes suspend workflow execution until an external signal arrives. Unlike the continuous inline suggestions of an AI Coding Assistant or the fully autonomous execution of an Autonomous Cloud Coding Agent, here operator review represents a first-class graph node reserved for sensitive or regulated actions.

• Triggers: Activated prior to outbound email dispatch, pull request merges, or high-risk tool operations.
• Reviewer UI: Presents artifact previews referenced directly from shared state, where approve or reject button clicks invoke the signaling endpoint.
• Timeouts: Applies a configurable 24-hour SLA to either escalate to an alternate reviewer or gracefully fail the run according to tenant policy.

6. Streaming and Observability

Operators watch multi-agent graphs execute in real time. Stale telemetry degrades trust in approval previews, requiring node transitions to stream with sub-second latency.

• Event stream: Emits granular events including node_started, state_patch, checkpoint, hitl_required, node_completed, and completed.
• Reconnection handling: Uses the Last-Event-ID header backed by a Redis stream buffer with a one-hour time-to-live to ensure gapless replay upon client reconnection.
• Cost attribution: Appends consumed tokens, computed dollar cost, and elapsed execution latency to every node_completed event for real-time tenant quota tracking.