Interview Setup
Interview Prompt
Design a Stripe style subscription billing platform for 50M active subscriptions, 1.7M daily renewals, 5K peak invoice TPS. Support plans, trials, proration, dunning, and merchant webhooks. Integrate with an external payment gateway for card charges.
Clarifying Questions (ask before designing)
| Question | Why it matters |
|---|---|
| One time payments vs recurring: do we build payment processing? | Recurring billing orchestrates charges but delegates card network calls to the Payment Gateway. We own subscription state, invoicing, and retry logic. |
| What billing models: flat rate, tiered, metered, or seat based? | Flat rate is simplest with fixed price per period. Metered adds usage aggregation pipelines. Seat based requires mid cycle quantity changes and proration. |
| How do plan changes mid cycle work? | Proration is critical billing logic: credit unused time on the old plan and charge the remainder on the new plan. Interviewers probe the math and race conditions. |
| What happens when a renewal payment fails? | Dunning smart retries and customer communication determine churn vs recovery. The workload assumption uses approximately 40% recovery for optimized retry timing. |
Scope
In scope
- Plan and price catalog management
- Subscription lifecycle (create, upgrade, downgrade, pause, resume, and cancel)
- Invoice generation with proration and tax line items
- Billing scheduler for recurring renewals
- Dunning and payment retry logic
- Webhook delivery to merchants
- Integration with external payment gateway
Out of scope (state explicitly)
- Building a payment processor or PSP integration layer (delegated to Payment Gateway)
- Tax calculation engine internals (integrate Stripe Tax or Avalara)
- Revenue recognition / ASC 606 accounting (mention as downstream)
- Merchant onboarding and KYC
Functional Requirements
Start by asking your interviewer whether you orchestrate recurring billing or also build payment processing. Clarify proration on mid cycle plan changes, dunning on failed renewals, and exactly once billing per period, because that invariant separates billing from a simple cron job that charges cards.
- Plan catalog: Merchants define products with recurring prices (monthly, annual), tiers, trial periods, and usage based meters
- Subscription management: Create, upgrade, downgrade, pause, resume, and cancel subscriptions with proration on mid cycle changes
- Invoice generation: Auto-generate invoices at each billing period with line item breakdown (base charge, proration credit/charge, tax, discounts)
- Payment collection: Create an initial payment attempt for each finalized invoice through Payment Gateway on invoice finalization
- Dunning: Smart retry schedule for failed payments with escalating customer communication
- Webhooks: Notify merchants of subscription lifecycle events (created, renewed, payment_failed, canceled)
- Customer portal: Self-service plan changes, payment method updates, invoice history
- Trials: Free trial periods that convert to paid subscriptions automatically
Non-Functional Requirements
Your interviewer will stress-test the requirement for at most one effective successful renewal charge per recurring billing period and complete auditability. Walk proration math with concrete mid cycle upgrade numbers when asked, because penny accuracy and an immutable event log backed by Event Sourcing and CQRS are what separate Stripe style billing from a simple scheduler.
- Exactly once billing effect: Each recurring renewal should produce at most one effective successful charge for its billing period. Provider duplicate captures remain durably recorded as non-effective successes and are corrected through an idempotent refund or void. Proration and adjustment invoices are separate logical charges and each is independently idempotent. Scheduler, invoice, and payment gateway idempotency plus reconciliation enforce the intended business effect.
- Auditability: Complete immutable history of every subscription state change and invoice amount calculation backed by Event Sourcing and CQRS.
- High Availability: 99.99% for subscription API. Billing batch tolerates 1h delay.
- Scalability: 50M active subscriptions, 5K peak invoice TPS during billing windows
- Correctness over speed: Proration math must be penny accurate. Rounding rules are documented and tested, and external payment effects are protected with idempotency and reconciliation rather than an unsupported promise of literal exactly once execution
- PCI compliance: Never store raw card data, delegating tokenization and sensitive cardholder data storage to the Payment Gateway.
Capacity Estimations
Size the billing scheduler before you size the API tier. Daily renewals and peak invoice TPS determine generator worker capacity and idempotency volume. Webhook volume drives Kafka partition count. The invoice storage estimate is a logical payload estimate that excludes indexes, replication, and operational metadata.
| Metric | Calculation | Value |
|---|---|---|
| Active subscriptions | Given | 50M |
| Billing cycles / day | 50M subs ÷ 30-day avg cycle | ~1.7M renewals/day |
| Average renewal rate | 1.7M renewals ÷ 86,400 sec/day | ~20 invoices/sec average |
| Peak billing TPS | Given (midnight UTC batch + proration spikes) | 5K invoices/sec |
| Peak to average renewal ratio | 5K ÷ ~20 | ~250x burst |
| Plans in catalog | Given | 10K (across all merchants) |
| Invoice line items / invoice | Given (avg) | 3 (base + proration + tax) |
| Invoice storage / day | 1.7M x 2 KB | ~3.4 GB/day before additional proration, adjustment invoices, indexes, replication, and operational metadata |
| Base invoice webhook events / day | 1.7M renewals x 3 events each | ~5M/day, excluding dunning, plan changes, and other lifecycle events |
| Dunning retries / day | ~3% failure rate x 3 average retries | ~150K retry attempts |
Billing scheduler assigns each subscription a stable billing_shard = hash(subscription_id) % 64 and workers process one shard at a time. Each of the 64 logical shards contains roughly 780K active subscriptions, or about 26K renewals/day on average. At 5K invoices/sec peak, need ~50 invoice generator workers assuming each worker sustains roughly 100 invoice generations/sec after accounting for Payment Gateway I/O, retries, and database work. PostgreSQL connection pooling (PgBouncer) prevents each worker from holding a dedicated database connection. Checkpoint cursors reduce rework after a crash, while invoice and payment idempotency remain the correctness mechanism when work is retried.
Architecture Diagram
Walk your interviewer through orchestration vs delegation first. Subscription billing orchestrates recurring charges but delegates card network processing to the Payment Gateway, executing one initial payment attempt per finalized invoice rather than keeping a standing authorization. Later dunning retries create distinct payment attempts with their own idempotency keys.
Subscription state is event sourced using Event Sourcing and CQRS for auditability, where every plan change, trial end, and payment failure appends to an immutable log. The domain transaction also writes an outbox record so the event can be published to Kafka without a database-to-broker gap. The billing scheduler drives the daily renewal batch, while dunning handles failures asynchronously without blocking the next customer's charge.
Structure the architecture into three lanes: catalog (plans and prices), lifecycle (subscription state machine), and money movement (invoice generation, payment gateway charge, and webhook confirmation). Proration and Idempotency live in the lifecycle lane, which is where interviewers spend most of their time.
In the room
State upfront: "We do not process cards directly because our payment gateway partner handles card processing. We own subscription state, invoicing, proration, and dunning." Then walk a mid cycle upgrade with real dollar math before they ask.
Subscription Lifecycle State Machine
Component Deep Dives
1. Plan and Price Catalog
Plans and prices represent the catalog layer, while the state machine and proration engine are where billing edge cases originate.
Immutable products and versioned prices allow existing subscribers to stay on grandfathered rates until an explicit migration. Invoice line items snapshot the effective price, quantity, currency, and tax inputs so later catalog changes cannot rewrite historical charges.
- Product: Logical offering (e.g., "Pro Plan"). Immutable once subscriptions exist on it.
- Price: Recurring amount, billing interval (monthly or annual), and currency. Price changes create new versioned Price records while existing subscriptions remain locked to their original price.
- Trial: Optional free period such as 7, 14, or 30 days. Subscriptions start in the TRIALING state, and the first invoice is generated only at trial expiration.
- Metered prices: Base unit amount multiplied by aggregated usage quantities measured across the billing period.
2. Subscription Lifecycle Manager
Every state transition updates the current subscription projection, appends an event to the subscription event stream backed by Event Sourcing and CQRS. The current active state is maintained as a projection of these events. Optimistic concurrency control using an expected version prevents lost updates during concurrent plan adjustments, while the state projection, event, and outbox records are committed atomically before Kafka publication.
Stream: subscription-sub_abc123
v1: {type: "SubscriptionCreated", plan: "pro_monthly", customer: "cus_xyz"}
v2: {type: "TrialStarted", trial_end: "2026-04-01"}
v3: {type: "TrialEnded", invoice_id: "inv_001"}
v4: {type: "InvoicePaid", invoice_id: "inv_001", amount: 3000}
v5: {type: "PlanChanged", old_plan: "pro_monthly", new_plan: "enterprise_monthly", proration_invoice: "inv_002"}
v6: {type: "PaymentFailed", invoice_id: "inv_003", attempt: 1}
v7: {type: "SubscriptionCanceled", reason: "payment_failed_max_retries", effective: "2026-05-01"}3. Proration Engine ⭐
Proration represents critical billing logic by crediting unused time on the old plan and charging remainder time on the new one, accompanied by itemized line items for customer transparency.
Customer on Pro ($30/mo), upgrades to Enterprise ($100/mo) on Day 15 of 30: Unused Pro credit: $30 x (15/30) = $15.00 credit Enterprise prorated: $100 x (15/30) = $50.00 charge Net invoice line items: - Credit: -$15.00 (unused Pro) - Charge: +$50.00 (Enterprise remainder) - Total: +$35.00 due immediately Implementation: 1. Snapshot current plan + period_start/end from subscription state 2. Compute daily rate = plan_amount / days_in_period 3. Credit unused days on old plan, charge remaining days on new plan 4. Generate invoice with line item breakdown (transparency for disputes)
Key edge cases include month end anchors (such as January 31 transitioning to February 28), leap years, currency specific minor unit rounding using decimal arithmetic, and simultaneous plan changes combined with quantity changes. The example uses USD cents and half up arithmetic. Perform calculations in integer minor units with decimal arithmetic rather than binary floating point. For multi currency prices, apply the configured minor unit scale and rounding rule for that currency. Always generate clear itemized line items so customers and finance audit teams have complete transparency.
4. Billing Scheduler
The daily renewal batch represents a throughput challenge where subscriptions are sharded and cursors are checkpointed. A scheduler crash can cause work to be retried, but idempotent invoice and payment operations prevent double billing.
- Daily batch or continuous scanner: query subscriptions WHERE
current_period_end <= NOW()AND status IN (ACTIVE, TRIALING) ANDcancel_at_period_end = false. - Shard by subscription ID across the worker pool. Each worker generates the invoice, finalizes it, and invokes the Payment Gateway to collect charges.
- A checkpoint table records the last processed shard cursor so recovery resumes near the failure point. Advance the checkpoint only after the invoice and payment attempt records are durably committed. Reprocessing can still occur, and idempotency on invoice and payment operations makes that replay safe.
- Load distribution: stagger billing execution by customer timezone or subscription ID hashing to avoid midnight UTC traffic spikes. A fenced scheduler lease assigns a monotonically increasing scheduler epoch, and each billing job carries that epoch so a stale scheduler cannot continue issuing work after failover.
5. Invoice and Payment Attempt State
Invoices are durable financial records, while payment attempts represent interactions with the external Payment Gateway. Keep their lifecycles separate so an uncertain gateway response does not corrupt invoice state. Finalize the invoice and persist its initial payment attempt in one database transaction, then call the external gateway outside that transaction. This leaves a durable payment attempt even if the process crashes before the provider call or before the provider response is recorded. A finalized invoice can have multiple payment attempts over dunning, but only one attempt can produce the effective successful payment for that invoice. Each attempt has its own idempotency key and remains in an unknown state until the provider outcome is resolved.
- Draft invoices become immutable financial records at finalization, with a price, quantity, currency, and tax snapshot preserved for audit.
- Invoice idempotency is scoped to the logical invoice operation. A recurring renewal is keyed by subscription and billing period, while proration and adjustment invoices use their source event or request key so multiple invoices may exist in one period.
- Payment attempts move through pending, unknown, succeeded, or failed. An unknown attempt blocks a new retry until reconciliation resolves it.
- Gateway webhooks and polling update payment and invoice state through optimistic concurrency. Duplicate provider events are ignored by provider_event_id, and a terminal state cannot be regressed by a stale event.
6. Dunning Manager
Failed renewals represent a recovery opportunity rather than immediate hard failures. Intelligent retry timing can recover a significant fraction of transient card declines. The workload estimate assumes three average retry attempts, while the default policy exposes four possible retry dates and can skip a date when payment state or decline type makes the attempt inappropriate.
- Configurable retry schedule per merchant, defaulting to retries on day 0, 3, 7, and 14.
- Smart retry heuristics: filter out hard declines and schedule retries within optimal card network processing windows to maximize recovery rates. Before creating a new retry attempt, reconcile any previous payment attempt that remains pending or unknown.
- Escalating communication cadence: initial email notification leading to in-app banners and optional SMS alerts.
- Grace period policy: keep subscriptions active or transition them to past-due status during active dunning attempts. Do not schedule a new attempt while a prior gateway attempt remains unresolved.
7. Webhook Delivery Service
Merchants provision application entitlements from webhooks, so delivery must be signed, idempotent, retryable, and safe under duplicates and reordering.
- Consumes subscription and invoice events from Kafka through a unified merchant delivery projection so webhook consumers do not depend on cross topic ordering. When strict ordering is part of the merchant contract, the delivery worker sends sequence N+1 only after sequence N has reached a terminal delivered or dead lettered state.
- Signs outgoing payloads with HMAC-SHA256 over the timestamp and exact serialized body, includes stable event IDs, and tracks delivery state per merchant endpoint so a retry cannot create a second logical delivery record.
- Retries failed deliveries using exponential backoff from 1 second up to 24 hours, routing permanently undeliverable messages to a dead letter queue.
- Accepts payment gateway webhooks through a separate verified ingress path that validates the provider signature, durably records the provider event ID, and applies invoice state changes idempotently.
8. Event Bus Design (Kafka)
Kafka decouples billing events from webhook delivery and downstream analytics. Subscription lifecycle events remain ordered per subscription by partitioning on subscription_id. Invoice events are partitioned by merchant_id for merchant-facing aggregation. A dedicated webhook projection can merge these sources into a merchant-facing stream, but consumers should still use event IDs and resource versions for deduplication and should not assume cross resource ordering unless the projection explicitly assigns a per-merchant sequence.
Topic: subscription-events Partitions: 64 (partition by subscription_id for ordered lifecycle events per sub) Retention: 90 days operational replay Events: subscription.created, plan.changed, trial.ended, paused, resumed, canceled Consumers: webhook projection, analytics, entitlement sync Topic: invoice-events Partitions: 32 (partition by merchant_id) Events: invoice.created, invoice.finalized, invoice.paid, invoice.payment_failed Consumers: dunning manager, merchant dashboard indexer, accounting export Ordering note: merchant_id ordering is useful for merchant-facing aggregation but can create a hot partition for a dominant merchant. Size partitions from measured peak load and change the keying strategy if one merchant becomes a significant share of traffic. Topic: merchant-webhook-events Partitions: keyed by merchant_id Purpose: unified merchant delivery stream. Events carry event_id and resource_version. Delivery workers allocate a monotonic delivery_sequence per endpoint when creating delivery records, and enforce sequence order only when that endpoint's contract requires strict ordering. Delivery attempt state is tracked separately for each endpoint. Consumers: Webhook Delivery workers Topic: billing-jobs Partitions: 64 (Kafka key = subscription_id, with logical shard = hash(subscription_id) % 64) Producers: Billing Scheduler (daily renewal batch) Consumers: Invoice Generator workers (horizontally scaled) Job fields: billing_run_id, subscription_id, scheduler_epoch, due_at Publication safety: the subscription or invoice database transaction writes the domain event and outbox record atomically. An outbox publisher or CDC pipeline publishes to Kafka with idempotent producer semantics. Publication can still be retried after an ambiguous publisher crash, so consumers deduplicate by event_id. Audit retention: keep the append only event store for 7 years or the required financial retention period. For this scale, partition long lived event tables by time or tenant shard and archive older partitions to durable low cost storage with immutability controls where required. Kafka is the operational replay layer. Rebuild longer term merchant webhook streams from the durable event store after Kafka retention expires. Idempotency: recurring renewal invoice creation uses (subscription_id, billing_period_start, invoice_kind=renewal). Proration and adjustment invoices use a distinct source event or request key so multiple invoices can legitimately occur within one billing period. Payment attempts use invoice_id + attempt_number for gateway idempotency. A retry of the same logical payment attempt reuses its key, while a later dunning attempt uses a new attempt number and key. Event store: append only subscription_events and invoice_events tables following Event Sourcing patterns
API Design
Service Interfaces and Domain Types
Use explicit domain types for subscription, pricing, payment, idempotency, and optimistic concurrency values so API contracts remain clear across services.
type SubscriptionId = string;
type CustomerId = string;
type PriceId = string;
type PaymentMethodId = string;
type InvoiceId = string;
type PaymentAttemptId = string;
type InvoiceKind = "renewal" | "proration" | "adjustment" | "manual";
type IdempotencyKey = string;
type CurrencyCode = string;
type ISODateTime = string;
type EventVersion = number;
type Quantity = number;
type MinorUnits = number;
type SubscriptionStatus = "trialing" | "active" | "past_due" | "paused" | "canceled" | "unpaid";
type ProrationBehavior = "create_prorations" | "none";
type PaymentAttemptStatus = "pending" | "unknown" | "succeeded" | "failed";
export interface CreateSubscriptionRequest {
customerId: CustomerId;
priceId: PriceId;
paymentMethodId: PaymentMethodId;
quantity?: Quantity;
trialPeriodDays?: number;
idempotencyKey: IdempotencyKey;
}
export interface ChangePlanRequest {
subscriptionId: SubscriptionId;
priceId: PriceId;
quantity?: Quantity;
prorationBehavior: ProrationBehavior;
expectedEventVersion: EventVersion;
idempotencyKey: IdempotencyKey;
}
export interface CancelSubscriptionRequest {
subscriptionId: SubscriptionId;
cancelAtPeriodEnd: boolean;
expectedEventVersion: EventVersion;
idempotencyKey: IdempotencyKey;
}
export interface PauseSubscriptionRequest {
subscriptionId: SubscriptionId;
effectiveAt?: ISODateTime;
expectedEventVersion: EventVersion;
idempotencyKey: IdempotencyKey;
}
export interface ResumeSubscriptionRequest {
subscriptionId: SubscriptionId;
expectedEventVersion: EventVersion;
idempotencyKey: IdempotencyKey;
}
export interface PaymentAttemptSummary {
paymentAttemptId: PaymentAttemptId;
status: PaymentAttemptStatus;
amount: MinorUnits;
currency: CurrencyCode;
providerPaymentId?: string;
}
export interface IdempotencyResult {
key: IdempotencyKey;
requestHash: string;
status: "processing" | "completed" | "failed";
resourceId?: SubscriptionId | InvoiceId | PaymentAttemptId;
responseBody?: unknown;
}
export interface SubscriptionResponse {
subscriptionId: SubscriptionId;
status: SubscriptionStatus;
currentPeriodStart: ISODateTime;
currentPeriodEnd: ISODateTime;
quantity: Quantity;
currency: CurrencyCode;
latestInvoiceId?: InvoiceId;
cancelAtPeriodEnd?: boolean;
canceledAt?: ISODateTime;
pausedAt?: ISODateTime;
resumeAt?: ISODateTime;
eventVersion: EventVersion;
}
export interface InvoiceSummary {
invoiceId: InvoiceId;
subscriptionId: SubscriptionId;
kind: InvoiceKind;
status: "draft" | "open" | "paid" | "void" | "uncollectible";
amountDue: MinorUnits;
amountPaid: MinorUnits;
currency: CurrencyCode;
}
export interface BillingService {
createSubscription(request: CreateSubscriptionRequest): Promise<SubscriptionResponse>;
getSubscription(subscriptionId: SubscriptionId): Promise<SubscriptionResponse>;
getInvoice(invoiceId: InvoiceId): Promise<InvoiceSummary>;
changePlan(request: ChangePlanRequest): Promise<SubscriptionResponse>;
cancelSubscription(request: CancelSubscriptionRequest): Promise<SubscriptionResponse>;
pauseSubscription(request: PauseSubscriptionRequest): Promise<SubscriptionResponse>;
resumeSubscription(request: ResumeSubscriptionRequest): Promise<SubscriptionResponse>;
}Create Subscription
Lead with idempotency because every mutating billing call carries an Idempotency-Key. When interviewers ask what happens if the scheduler retries a renewal, explain that the recurring invoice key is scoped to the subscription, billing period, and invoice kind. Proration and adjustment flows use their own logical keys so a legitimate mid cycle invoice cannot collide with the recurring renewal. Payment retries use a separate key per attempt, and a retry of the same logical attempt reuses that key. The API idempotency record protects request replay within its retention window, while the durable invoice and payment keys enforce the longer lived billing invariant.
POST /api/v1/subscriptions
Idempotency-Key: idem-sub-uuid-12345
Authorization: Bearer <merchant_api_key>
{
"customer_id": "cus_xyz",
"price_id": "price_pro_monthly",
"payment_method_id": "pm_card_visa",
"quantity": 1,
"trial_period_days": 14,
"metadata": {"source": "website_signup"}
}
Response: 201 Created
{
"subscription_id": "sub_abc123",
"status": "trialing",
"current_period_start": "2026-03-15T00:00:00Z",
"current_period_end": "2026-03-29T00:00:00Z",
"trial_end": "2026-03-29T00:00:00Z"
}Get Subscription
GET /api/v1/subscriptions/sub_abc123
Authorization: Bearer <merchant_api_key>
Response: 200 OK
{
"subscription_id": "sub_abc123",
"status": "active",
"current_period_start": "2026-03-29T00:00:00Z",
"current_period_end": "2026-04-29T00:00:00Z",
"quantity": 1,
"currency": "USD",
"event_version": 8
}Get Invoice
GET /api/v1/invoices/inv_002
Authorization: Bearer <merchant_api_key>
Response: 200 OK
{
"invoice_id": "inv_002",
"subscription_id": "sub_abc123",
"status": "paid",
"amount_due": 3500,
"amount_paid": 3500,
"currency": "USD"
}Change Plan (with Proration)
POST /api/v1/subscriptions/sub_abc123/plan-change
Idempotency-Key: idem-plan-change-uuid
If-Match: "7"
{
"price_id": "price_enterprise_monthly",
"proration_behavior": "create_prorations"
}
Response: 200 OK
{
"subscription_id": "sub_abc123",
"status": "active",
"event_version": 8,
"latest_invoice": {
"invoice_id": "inv_002",
"amount_due": 3500,
"line_items": [
{"description": "Unused time on Pro", "amount": -1500},
{"description": "Remaining time on Enterprise", "amount": 5000}
]
}
}Cancel Subscription
POST /api/v1/subscriptions/sub_abc123/cancel
Idempotency-Key: idem-cancel-uuid
If-Match: "8"
{
"cancel_at_period_end": true
}
Response: 200 OK
{
"subscription_id": "sub_abc123",
"status": "active",
"cancel_at_period_end": true,
"canceled_at": null,
"current_period_end": "2026-04-15T00:00:00Z"
}Pause and Resume Subscription
POST /api/v1/subscriptions/sub_abc123/pause
Idempotency-Key: idem-pause-uuid
If-Match: "8"
{
"effective_at": "2026-04-01T00:00:00Z"
}
Response: 200 OK
{
"subscription_id": "sub_abc123",
"status": "paused",
"event_version": 9
}
POST /api/v1/subscriptions/sub_abc123/resume
Idempotency-Key: idem-resume-uuid
If-Match: "9"
Response: 200 OK
{
"subscription_id": "sub_abc123",
"status": "active",
"event_version": 10
}Webhook Event
POST https://merchant.com/webhooks/billing
Billing-Signature: t=1710000000,v1=abc123...
{
"event_id": "evt_uuid",
"type": "invoice.payment_failed",
"data": {
"invoice_id": "inv_003",
"subscription_id": "sub_abc123",
"amount_due": 3000,
"attempt_count": 2,
"next_payment_attempt": "2026-03-18T00:00:00Z"
}
}Common Error Responses
400 Bad Request: invalid input, missing required fields, or malformed JSON payload 401 Unauthorized: missing or invalid authentication token or API key 403 Forbidden: authenticated caller lacks required permissions for this resource 404 Not Found: requested resource ID does not exist 409 Conflict: duplicate write or version conflict, retry with a unique idempotency key 422 Unprocessable Entity: syntactically valid request failed semantic business validation 429 Too Many Requests: rate limit quota exceeded, client should honor Retry-After header 500 Internal Error: unexpected server failure, retry safely with an idempotency key 503 Service Unavailable: downstream dependency is unavailable or overloaded, retry with exponential backoff 402 Payment Required: account balance or payment method has insufficient funds 502 Bad Gateway: payment gateway provider timeout, poll transaction status endpoint
Data Model
PostgreSQL: Plans & Prices
CREATE TABLE products (
product_id UUID PRIMARY KEY,
merchant_id UUID NOT NULL,
name VARCHAR(256) NOT NULL,
tax_code VARCHAR(64),
active BOOLEAN DEFAULT true,
created_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE prices (
price_id UUID PRIMARY KEY,
product_id UUID NOT NULL REFERENCES products(product_id),
unit_amount BIGINT NOT NULL, -- cents
currency VARCHAR(3) NOT NULL,
billing_interval VARCHAR(16) NOT NULL, -- 'month', 'year'
interval_count INT DEFAULT 1,
trial_period_days INT DEFAULT 0,
billing_scheme VARCHAR(16) DEFAULT 'per_unit', -- 'per_unit', 'tiered', 'metered'
active BOOLEAN DEFAULT true,
created_at TIMESTAMPTZ NOT NULL
);PostgreSQL: Subscriptions
CREATE TABLE subscriptions (
subscription_id UUID PRIMARY KEY,
merchant_id UUID NOT NULL,
customer_id UUID NOT NULL,
price_id UUID NOT NULL REFERENCES prices(price_id),
status VARCHAR(20) NOT NULL, -- trialing, active, past_due, paused, canceled, unpaid
current_period_start TIMESTAMPTZ NOT NULL,
current_period_end TIMESTAMPTZ NOT NULL,
trial_end TIMESTAMPTZ,
cancel_at_period_end BOOLEAN DEFAULT false,
canceled_at TIMESTAMPTZ,
billing_anchor_day SMALLINT,
billing_anchor_time TIME NOT NULL DEFAULT '00:00:00',
billing_timezone VARCHAR(64) NOT NULL DEFAULT 'UTC',
billing_shard SMALLINT NOT NULL, -- hash(subscription_id) % 64
paused_at TIMESTAMPTZ,
resume_at TIMESTAMPTZ,
default_payment_method VARCHAR(64),
event_version INT NOT NULL DEFAULT 0, -- optimistic concurrency (event sourcing)
created_at TIMESTAMPTZ NOT NULL,
updated_at TIMESTAMPTZ NOT NULL
);
CREATE INDEX idx_subscriptions_billing
ON subscriptions(billing_shard, status, current_period_end, subscription_id);
CREATE INDEX idx_subscriptions_customer ON subscriptions(customer_id);PostgreSQL: Invoices
CREATE TABLE invoices (
invoice_id UUID PRIMARY KEY,
subscription_id UUID NOT NULL,
merchant_id UUID NOT NULL,
customer_id UUID NOT NULL,
status VARCHAR(20) NOT NULL, -- draft, open, paid, void, uncollectible
amount_due BIGINT NOT NULL,
amount_paid BIGINT NOT NULL DEFAULT 0,
currency VARCHAR(3) NOT NULL,
period_start TIMESTAMPTZ NOT NULL,
period_end TIMESTAMPTZ NOT NULL,
price_snapshot_json JSONB NOT NULL, -- immutable effective price, quantity, currency, and tax inputs
effective_payment_attempt_id UUID, -- populated only for the effective successful payment
invoice_kind VARCHAR(32) NOT NULL, -- renewal, proration, adjustment, manual
source_event_id UUID, -- source event when applicable, such as a plan change
idempotency_key VARCHAR(128) NOT NULL UNIQUE, -- stable key for the logical invoice operation
due_date TIMESTAMPTZ,
finalized_at TIMESTAMPTZ,
paid_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL
);
-- A subscription may have multiple legitimate invoices within one billing period,
-- such as a recurring renewal plus one or more proration or adjustment invoices.
-- The logical operation key above provides idempotency without blocking those cases.
CREATE INDEX idx_invoices_subscription ON invoices(subscription_id, period_start);
CREATE TABLE invoice_line_items (
line_item_id UUID PRIMARY KEY,
invoice_id UUID NOT NULL REFERENCES invoices(invoice_id),
description TEXT NOT NULL,
amount BIGINT NOT NULL, -- negative for credits, stored in minor units
unit_amount BIGINT NOT NULL,
quantity INT NOT NULL DEFAULT 1,
currency VARCHAR(3) NOT NULL,
price_id UUID,
tax_code VARCHAR(64),
discount_amount BIGINT NOT NULL DEFAULT 0,
tax_amount BIGINT NOT NULL DEFAULT 0
);
CREATE TABLE payment_attempts (
payment_attempt_id UUID PRIMARY KEY,
invoice_id UUID NOT NULL REFERENCES invoices(invoice_id),
attempt_number INT NOT NULL,
gateway_idempotency_key VARCHAR(128) NOT NULL UNIQUE,
provider_payment_id TEXT,
provider_status VARCHAR(64),
status VARCHAR(20) NOT NULL, -- pending, unknown, succeeded, failed
is_effective BOOLEAN NOT NULL DEFAULT false,
failure_code VARCHAR(64),
created_at TIMESTAMPTZ NOT NULL,
completed_at TIMESTAMPTZ,
UNIQUE (invoice_id, attempt_number)
);
-- Provider duplicate captures can both be recorded as succeeded, but only one
-- attempt may be marked effective for a given invoice.
CREATE UNIQUE INDEX idx_one_effective_success_per_invoice
ON payment_attempts(invoice_id)
WHERE status = 'succeeded' AND is_effective = true;
-- Keep the invoice's effective payment linked to a payment attempt for the same invoice.
CREATE UNIQUE INDEX idx_payment_attempt_invoice_pair
ON payment_attempts(payment_attempt_id, invoice_id);
ALTER TABLE invoices
ADD CONSTRAINT fk_invoice_effective_payment_attempt
FOREIGN KEY (invoice_id, effective_payment_attempt_id)
REFERENCES payment_attempts(invoice_id, payment_attempt_id);
CREATE TABLE gateway_events (
provider_event_id TEXT PRIMARY KEY,
event_type VARCHAR(64) NOT NULL,
payment_attempt_id UUID REFERENCES payment_attempts(payment_attempt_id),
invoice_id UUID REFERENCES invoices(invoice_id),
payload_json JSONB NOT NULL,
received_at TIMESTAMPTZ NOT NULL,
processing_started_at TIMESTAMPTZ,
lease_expires_at TIMESTAMPTZ,
processed_at TIMESTAMPTZ,
status VARCHAR(20) NOT NULL -- received, processing, processed, ignored
);
CREATE INDEX idx_gateway_events_processing ON gateway_events(status, received_at, processing_started_at);
CREATE TABLE payment_refunds (
refund_id UUID PRIMARY KEY,
invoice_id UUID NOT NULL REFERENCES invoices(invoice_id),
payment_attempt_id UUID NOT NULL REFERENCES payment_attempts(payment_attempt_id),
gateway_refund_id TEXT,
refund_idempotency_key VARCHAR(128) NOT NULL UNIQUE,
amount BIGINT NOT NULL,
currency VARCHAR(3) NOT NULL,
reason VARCHAR(64) NOT NULL, -- duplicate_capture, customer_request, billing_correction
status VARCHAR(20) NOT NULL, -- pending, succeeded, failed
created_at TIMESTAMPTZ NOT NULL,
completed_at TIMESTAMPTZ
);
CREATE INDEX idx_payment_refunds_invoice ON payment_refunds(invoice_id, created_at);
CREATE TABLE dunning_attempts (
dunning_attempt_id UUID PRIMARY KEY,
invoice_id UUID NOT NULL REFERENCES invoices(invoice_id),
attempt_number INT NOT NULL,
payment_attempt_id UUID REFERENCES payment_attempts(payment_attempt_id),
scheduled_at TIMESTAMPTZ NOT NULL,
started_at TIMESTAMPTZ,
status VARCHAR(20) NOT NULL, -- scheduled, running, succeeded, failed, skipped
failure_code VARCHAR(64),
created_at TIMESTAMPTZ NOT NULL,
UNIQUE (invoice_id, attempt_number)
);PostgreSQL: Usage Records
CREATE TABLE usage_records (
usage_record_id UUID PRIMARY KEY,
subscription_id UUID NOT NULL REFERENCES subscriptions(subscription_id),
metric_name VARCHAR(128) NOT NULL,
quantity NUMERIC(20,6) NOT NULL,
occurred_at TIMESTAMPTZ NOT NULL,
ingested_at TIMESTAMPTZ NOT NULL,
idempotency_key VARCHAR(128) NOT NULL UNIQUE
);
CREATE INDEX idx_usage_records_period
ON usage_records(subscription_id, metric_name, occurred_at);
CREATE TABLE usage_aggregates (
subscription_id UUID NOT NULL REFERENCES subscriptions(subscription_id),
metric_name VARCHAR(128) NOT NULL,
period_start TIMESTAMPTZ NOT NULL,
period_end TIMESTAMPTZ NOT NULL,
quantity NUMERIC(20,6) NOT NULL,
cutoff_at TIMESTAMPTZ,
finalized_at TIMESTAMPTZ,
PRIMARY KEY (subscription_id, metric_name, period_start)
);PostgreSQL: API Idempotency
CREATE TABLE api_idempotency_keys (
idempotency_key VARCHAR(128) NOT NULL,
merchant_id UUID NOT NULL,
request_hash TEXT NOT NULL,
status VARCHAR(20) NOT NULL, -- processing, completed, failed
resource_type VARCHAR(32), -- subscription, invoice, payment_attempt
resource_id UUID,
response_json JSONB,
processing_started_at TIMESTAMPTZ,
lease_expires_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL,
expires_at TIMESTAMPTZ NOT NULL,
PRIMARY KEY (merchant_id, idempotency_key)
);
-- A processing record is reclaimed only after lease_expires_at. The request hash must
-- match a retry, otherwise return an idempotency conflict. A completed record stores the
-- original response so retries reproduce the same result. Internal billing operations also
-- retain their durable business idempotency keys beyond this request replay window.PostgreSQL: Subscription Event Log
CREATE TABLE invoice_events (
event_id UUID PRIMARY KEY,
invoice_id UUID NOT NULL,
event_type VARCHAR(64) NOT NULL,
event_data JSONB NOT NULL,
version INT NOT NULL, -- monotonic per invoice
created_at TIMESTAMPTZ NOT NULL,
UNIQUE (invoice_id, version)
);
CREATE TABLE subscription_events (
event_id UUID PRIMARY KEY,
subscription_id UUID NOT NULL,
event_type VARCHAR(64) NOT NULL,
event_data JSONB NOT NULL,
version INT NOT NULL, -- monotonic per subscription
created_at TIMESTAMPTZ NOT NULL,
UNIQUE (subscription_id, version)
);
CREATE TABLE billing_scheduler_leases (
lease_name VARCHAR(64) PRIMARY KEY,
scheduler_epoch BIGINT NOT NULL,
owner_region VARCHAR(64) NOT NULL,
owner_id VARCHAR(128) NOT NULL,
lease_expires_at TIMESTAMPTZ NOT NULL,
updated_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE billing_scheduler_checkpoints (
billing_run_id UUID NOT NULL,
shard_id INT NOT NULL,
scheduler_epoch BIGINT NOT NULL, -- fencing epoch for the active scheduler leader
cursor_subscription_id UUID,
updated_at TIMESTAMPTZ NOT NULL,
PRIMARY KEY (billing_run_id, shard_id)
);
CREATE TABLE billing_outbox (
outbox_id UUID PRIMARY KEY,
aggregate_type VARCHAR(32) NOT NULL, -- subscription or invoice
aggregate_id UUID NOT NULL,
event_id UUID NOT NULL UNIQUE,
topic VARCHAR(128) NOT NULL,
payload_json JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL,
published_at TIMESTAMPTZ,
attempt_count INT NOT NULL DEFAULT 0,
next_attempt_at TIMESTAMPTZ
);
CREATE INDEX idx_billing_outbox_pending ON billing_outbox(published_at, next_attempt_at);
CREATE TABLE merchant_webhook_endpoints (
endpoint_id UUID PRIMARY KEY,
merchant_id UUID NOT NULL,
callback_url TEXT NOT NULL,
secret_ref VARCHAR(128) NOT NULL,
active BOOLEAN NOT NULL DEFAULT true,
created_at TIMESTAMPTZ NOT NULL
);
CREATE INDEX idx_webhook_endpoints_merchant ON merchant_webhook_endpoints(merchant_id, active);
CREATE TABLE webhook_endpoint_sequences (
endpoint_id UUID PRIMARY KEY REFERENCES merchant_webhook_endpoints(endpoint_id),
next_sequence BIGINT NOT NULL
);
CREATE TABLE webhook_deliveries (
event_id UUID NOT NULL, -- globally unique source event ID
merchant_id UUID NOT NULL,
endpoint_id UUID NOT NULL REFERENCES merchant_webhook_endpoints(endpoint_id),
delivery_sequence BIGINT NOT NULL, -- allocated transactionally per endpoint
status VARCHAR(20) NOT NULL, -- pending, delivered, dead_lettered
attempt_count INT NOT NULL DEFAULT 0,
next_attempt_at TIMESTAMPTZ,
last_error TEXT,
delivered_at TIMESTAMPTZ,
PRIMARY KEY (event_id, endpoint_id),
UNIQUE (endpoint_id, delivery_sequence)
);
CREATE INDEX idx_webhook_deliveries_due ON webhook_deliveries(endpoint_id, status, next_attempt_at);Fault Tolerance
Fault Tolerance Scenarios
| Concern | Solution |
|---|---|
| Double billing on scheduler retry | Use a deterministic recurring renewal idempotency key derived from (subscription_id, billing_period_start, invoice_kind), enforce it with a database uniqueness constraint, append the InvoiceGenerated event with expected version compare and swap, and use a payment scoped idempotency key at the Payment Gateway. |
| Payment succeeds but invoice not marked paid | Treat the Payment Gateway payment state as authoritative. Verified gateway webhooks are the primary update path, while provider status lookup or reconciliation covers missed webhooks. A reconciliation job matches provider payment IDs to payment attempts and invoices every 15 minutes, then repairs invoice state through the same concurrency controlled transition. |
| Proration race during plan change | Serialize the subscription update with expected version compare and swap or a row lock. Compute proration from the same committed subscription snapshot, including effective price and quantity. Create the invoice draft and state transition in one database transaction. |
| Payment Gateway timeout after charge request | Keep the payment attempt in an unknown state. Reuse the same gateway idempotency key for the same logical attempt while querying or reconciling the provider. Do not create a new attempt until the earlier attempt is definitively failed or resolved. If the provider idempotency window has expired, resolve the provider payment status before sending a new charge, and only create a new attempt when the provider confirms that no charge was created. |
| Two payment attempts both appear successful | Persist every provider result, but allow only one payment attempt to become the effective successful payment for an invoice. If reconciliation discovers that more than one provider payment actually captured funds, keep the extra attempt as succeeded but non-effective, mark it as an overpayment case, issue a provider refund or void using a stable refund idempotency key, and record the correction in the invoice, refund, and event streams. |
| Webhook replay or forged payment event | Verify the provider signature against the raw request body and a bounded timestamp window. Persist provider_event_id with idempotent processing state before applying the transition, ignore duplicates, and rotate webhook secrets through the secret manager. |
| Billing scheduler crash mid batch | Use a checkpointed shard cursor to resume work. Reprocessing is safe because invoice creation and payment submission are idempotent. The checkpoint reduces rework but is not the correctness guarantee. |
| Webhook delivery failure to merchant | Publish from a transactional outbox or CDC path into a Kafka delivery topic, persist delivery state per merchant endpoint, retry with exponential backoff from 1 second up to 24 hours, and route permanent failures to a dead letter queue. Merchants can poll GET /invoices/{id} as a fallback. |
| Timezone edge cases (month end billing) | Store the billing anchor as a local calendar rule with an IANA billing timezone, derive execution timestamps in UTC, clamp an anchor day such as 31 to the last valid day of a shorter month, and persist the resolved period boundaries on the subscription or invoice. |
Additional Considerations
Relationship to Payment Gateway
The Payment Gateway handles one time card charges such as authorize, capture, refund, and idempotency keys per payment request. The subscription billing platform orchestrates when to charge across billing periods, proration, and dunning retries. Each chargeable invoice finalization creates a payment attempt with a payment scoped idempotency key before calling the Payment Gateway. Billing workers respect provider rate limits with bounded concurrency and backpressure. A provider 429 response reschedules the existing attempt rather than creating a new attempt, and circuit breaking prevents an outage from generating a retry storm. If the gateway response times out, the platform treats the attempt as unresolved and reuses the same key for provider status lookup or retry while it remains valid until the provider outcome is known. Subscription state lives in the billing platform, while payment execution state lives in the Payment Gateway. Periodic reconciliation jobs match provider payment IDs with invoice and payment attempt records.
Payment Gateway Webhook Ingress
Payment completion is confirmed asynchronously by the external gateway. The inbound webhook path verifies authenticity, persists the provider event ID before applying the state transition, and treats duplicate deliveries as safe retries. Invoice state changes then flow back through the same event and outbox path used by other durable billing events.
POST /internal/payment-gateway/webhooks HTTP/1.1
Content-Type: application/json
Payment-Signature: t=1710000000,v1=abc123...
{
"provider_event_id": "pevt_001",
"type": "payment.succeeded",
"payment_attempt_id": "pat_001",
"invoice_id": "inv_003",
"amount": 3500,
"currency": "USD"
}
Processing:
1. Verify the provider signature against the raw request body and enforce a bounded timestamp window.
2. Insert provider_event_id into gateway_events idempotently with status=processing, processing_started_at, and lease_expires_at. Only one consumer may win processing for that event.
3. If the event is already processed, return 200. If it is recorded as processing but its lease has expired, reclaim it with a conditional update on the current lease timestamp so only one worker owns the processing lease. Resume the unfinished processing safely. Mark the event processed in the same transaction as the payment state transition and outbox write.
4. Validate payment_attempt_id and invoice_id linkage, including the expected amount and currency.
5. Ignore stale or out of order events that would regress a terminal payment state, such as succeeded back to failed.
6. For a successful payment, atomically claim the invoice's effective payment slot if it is still empty. If another attempt is already effective, persist this attempt as succeeded but non-effective and create a durable refund or void request. For a failure, append PaymentFailed only when the attempt is not already terminally succeeded. A stale event cannot move a terminal payment attempt backward.
7. Publish the resulting internal event through the outbox path in the same database transaction as the state transition.Webhook Security
Webhook endpoints accept merchant controlled URLs, so delivery workers must validate destination configuration and use controlled egress. Store signing secrets in a secret manager, sign the exact serialized payload with a timestamp, enforce replay windows, and prevent DNS rebinding or access to private network addresses when resolving callback URLs. These controls protect both merchant data and the billing platform from forged events and unintended internal network access.
Credit Notes and Refunds
Billing corrections should create explicit credit notes or refund requests rather than mutating a finalized invoice amount in place. The billing platform records the adjustment, links it to the original invoice and payment, and calls the Payment Gateway with a stable refund idempotency key when money must be returned to the customer.
Revenue Recognition
Annual subscriptions paid upfront ($1200/year) require deferred revenue recognition of $100/month over 12 months. The downstream accounting system consumes invoice-events to create recognition schedules. This remains out of scope for the billing platform but should be mentioned in staff interviews.
Tax Calculation
Integrate Stripe Tax, Avalara, or TaxJar for jurisdiction-aware tax line items. Tax is calculated at invoice finalization from the customer address, product tax code, and applicable tax rules. Store tax_amount as a separate line item for audit. If the tax service is unavailable or returns an ambiguous result, keep the invoice in draft and do not submit a payment attempt until tax inputs are resolved.
Interview Walkthrough
- 25-minute cut
Skip the deeper staff-level trade-offs unless the interview reaches that level.
- Clarify scope: orchestrate billing while delegating card charges (5 min)
- Model subscription as an event sourced state machine (6 min)
- Enforce idempotency across scheduler, invoices, and payments (5 min)
- Walk through proration math with concrete numbers (5 min)
- Design dunning as a configurable retry schedule (4 min)
- Clarify scope: we orchestrate recurring billing, while card processing is delegated to the Payment Gateway via one time charges per invoice.
- Model subscription as an event sourced state machine backed by Event Sourcing and CQRS with an append only log, optimistic concurrency, and a complete audit trail.
- Enforce idempotency at three layers: the scheduler period key, invoice creation, and payment gateway charge submission.
- Walk through proration math with concrete numbers because interviewers look for mid cycle upgrade scenarios.
- Design dunning as a configurable retry schedule with escalating communication rather than a single retry.
- Billing scheduler: shard subscriptions, checkpoint cursors, and stagger billing times to avoid thundering herd spikes.
- Webhooks: signed, idempotent, and retried with backoff so merchants can reliably provision entitlements from these events.
- Common pitfall: treating subscription billing like a simple cron and charge loop without proration, dunning, or exactly once guarantees.
Engineering Trade-offs
Event Sourcing vs CRUD for Subscription State
CRUD operations like UPDATE subscriptions SET status = 'past_due' are simpler to implement but lose historical context. Billing disputes require reconstructing what plan was active on a specific historical date from audit logs. Event sourcing appends immutable events where current state is computed as a projection. The trade-off involves additional storage and slightly more complex read patterns, such as folding events or maintaining materialized views. For billing platforms, this complete audit trail is essential because financial regulators and accounting teams demand exact historical provenance.
Batch Billing vs Continuous Billing
Batch billing charges all due subscriptions in scheduled windows, which is simpler to reason about and gives predictable load spikes around the 5K TPS peak. Continuous billing charges each subscription near its exact anniversary, which smooths load but increases scheduler complexity. Hash based staggering can smooth the load further. For interviews, batch processing with sharding is the pragmatic starting point.
Immediate Cancel vs Cancel at Period End
Immediate cancellation simplifies the state machine, but the customer can lose paid for time and require refunds. Cancel at period end (⭐) lets the customer retain access until current_period_end. Apply a prorated refund when the product policy or applicable regulation requires it. The choice is a product and contract decision, so the platform should make the policy explicit rather than hard-coding one default.
Grace Period During Dunning vs Hard Cutoff
A hard cutoff transitions the subscription to PAST_DUE and revokes access immediately. This reduces revenue leakage but can increase churn from transient card failures. A grace period keeps access active during dunning retries and can improve recovery. The original workload assumption uses roughly 40% recovery for optimized retries. The behavior is configurable per merchant, and the entitlement service listens to subscription events to gate or restore access.
Building vs Buying Payment Processing
Avoid building a custom payment service provider during a billing system design interview. Delegate card network integration, PCI compliance, and 3D Secure verification to the Payment Gateway. Focus your design on subscription orchestration, proration, dunning, and webhook delivery, which is where core billing platform complexity resides.
Review
How helpful was this walkthrough?
Click a star to rate. We actively use this feedback to refine and update our system design content.
Discussion
Share your thoughts, ask questions, or help others.