System Design Interview: Design an E-commerce Payment Processing System

Prompt and Applicable Context

Design an e-commerce payment processing system. After a shopper submits an order, the system uses an external payment service provider to process a one-time card payment. It authorizes at checkout, captures after inventory is reserved, and releases the authorization if the order is canceled. After capture, it supports multiple partial refunds, but their cumulative amount must never exceed the captured amount. A payment method may require extra authentication such as 3DS, and the final outcome may arrive after the browser returns.

Assume five million payment attempts per day, about 58 TPS on average and 500 TPS at peak. The create-payment API has a p99 under 300 milliseconds, status reads have a p99 under 200 milliseconds, and the local API has a 99.99% monthly availability target. That availability does not promise provider completion. Money is an integer in the currency's minor unit, and one payment has exactly one currency. The provider, network, and local processes can all fail. These numbers and deadlines are interview assumptions, not promises made by a payment product.

The scope includes payment creation, extra authentication, authorization, capture, authorization cancellation, partial and full refunds, the money ledger, provider webhooks, and reconciliation. Chargebacks, a fraud model, foreign exchange, merchant payouts, tax, and a complete PCI compliance program are out of scope, but the answer should identify those boundaries. The provider's hosted page or tokenization component collects card details. This system stores a payment method token and must not receive or log the raw card number or security code.

What the Interviewer Evaluates

The first signal is whether the candidate separates business intent, provider attempts, and money facts. One cart maps to one internal payment_id, but it can have multiple authentication or provider attempts. A timed-out request also does not imply a failed payment. A strong answer does not compress the whole lifecycle into paid=true. It stores the payment aggregate, every operation and unknown result, provider references, and ledger entries separately.

The second signal is accurate reasoning across a distributed boundary. A database transaction cannot atomically include an external payment provider. The provider might capture while its response is lost, a webhook might arrive before the synchronous response, or a process might crash after a local commit. Caller idempotency keys, internal operation IDs, provider idempotency keys, a transactional outbox, webhook deduplication, and status lookup make repeated execution converge. They do not create an end-to-end exactly-once transaction.

The third signal is a monotonic, auditable state machine. Authorization and capture are different money stages. REQUIRES_ACTION is not a failure, and a provider timeout producing an UNKNOWN operation cannot immediately fail the payment. A refund must refer to a captured payment, use exact money, and preserve the invariant that confirmed plus in-flight refunds do not exceed the captured amount under concurrency.

The fourth signal is the division between workflow state and accounting. The payment table answers what the user can do next. An append-only ledger answers why a balance has its current value. Posted facts are never edited in place; refunds and corrections add reversing or compensating entries. Each journal's debits, credits, currency, and business reference are validated transactionally and then reconciled with provider reports and bank deposits.

The final signal is security and falsifiability. The answer should reduce card-data exposure, verify webhook signatures, protect provider secrets, constrain permissions and logs, and inject lost responses, duplicate or out-of-order webhooks, concurrent captures and refunds, unbalanced journals, and reconciliation discrepancies. A service diagram without invariants, recovery paths, and verification does not demonstrate correctness.

Questions to Clarify Before Answering

• Who collects the card data? This prompt uses the provider's hosted page or tokenization component. The business

backend receives only a payment method token and does not store a raw card number, track data, PIN, or security code. A compliance team still has to confirm the actual PCI scope.

• When do authorization and capture happen? Authorize at order submission and capture after inventory reservation.

Cancel before expiry if inventory fails. A capability table determines whether a payment method supports delayed capture; the design cannot assume that all methods do.

• What establishes success? A browser redirect is only a user-experience signal and cannot authorize fulfillment.

An authenticated provider response, webhook, or active query supplies authoritative evidence, which the local state machine must accept before a business action occurs.

• What is the refund contract? Support several partial refunds and a full refund, never exceeding capture. There are

no standalone refunds without an original payment. A refund can complete asynchronously or be rejected by the provider.

• Do we need multiple providers? Version one has one provider, but the interface stores an internal operation ID and

provider reference. Never fail over automatically while an outcome is unknown, because both providers could charge.

• What does the ledger cover? This prompt records processor receivable and merchant payable. Provider fees, merchant

payouts, and tax are out of scope. Every currency balances independently; no floating-point money or implicit exchange rate is allowed in the ledger.

• What are the retention and audit requirements? Retain payments, operations, webhook receipts, and ledger references

according to regulatory and company policy, while minimizing, encrypting, and auditing access to sensitive payloads. PCI SSC prohibits storing sensitive authentication data after authorization, even when encrypted.

• How do availability and consistency trade off? If the provider is unavailable, the system can accept work and show

“processing,” but it cannot show false success. Money correctness and traceability take priority over an immediate terminal response.

30-Second Answer Framework

“I would use a stable payment_id for one payment intent and separate payment state from authorize, capture, and refund operations. Caller idempotency keys atomically store the request digest, operation, and outbox; the worker reuses the operation ID at the provider. A timeout becomes UNKNOWN and converges through verified webhooks, lookup, and reconciliation rather than a fresh charge. Authoritative results advance a monotonic state machine and atomically append a balanced journal and business event. Refund creation conditionally reserves remaining refundable value. Provider reports and bank deposits then reconcile the internal ledger. Provider tokenization keeps card data out of the backend, and a browser success page cannot trigger fulfillment.”

Step-by-Step Deep Dive

Step 1: Start with capacity, invariants, and ownership

Five million divided by 86,400 seconds is about 58 TPS. A 500 TPS peak does not require every table to be sharded on day one. Correctness, auditability, and recovery across an external boundary dominate this design. The payment API scales horizontally, a relational database holds authoritative state, and a durable queue routed by payment_id absorbs peaks and isolates provider latency. State the invariants first:

amount > 0
currency is immutable after the first provider attempt
captured_amount <= authorized_amount
refunded_amount + pending_refund_amount <= captured_amount
for every journal: sum(debits) == sum(credits), per currency
one merchant + one idempotency_key describes one immutable request intent
one successful business operation produces at most one journal reference

At this scale, a relational database with one write region and failover makes idempotency keys, state versions, and ledger ordering easier to preserve than active-active multi-region writes. Active-active can reduce regional failover time, but it must resolve concurrent use of the same key and conflicting money operations; it is justified only by an explicit regional availability requirement. Full event sourcing also preserves history but spreads replay, schema migration, and query complexity across the payment workflow. This design keeps only money journals append-only and uses a versioned current aggregate for workflow state, matching the audit benefit to the operational cost.

The order service owns inventory and fulfillment. The payment service owns payment state, operations, and references to money facts. The provider owns the card-network state. The ledger owns internal money facts. The order service cannot write payment rows directly, and a payment webhook cannot directly mark an order shipped. The payment service publishes stable business events that the order service consumes idempotently by payment_id.

Step 2: Define the APIs, idempotency session, and data model

The core interface can be:

POST /payments                 create one payment for an order
POST /payments/{id}/capture    capture an authorized amount
POST /payments/{id}/cancel     cancel an uncaptured authorization
POST /payments/{id}/refunds    request a partial or full refund
GET  /payments/{id}            return current status and allowed next actions
POST /provider/webhooks        persist a verified provider event

Every money-changing call requires a caller-provided idempotency key. A unique constraint on (merchant_id, operation_type, idempotency_key) protects a normalized request digest, in-progress state, and replayable response. The first request creates payments, payment_operations, and an outbox record in one database transaction. The same key and digest replay the known response. The same key with a different amount, currency, payment, or operation type returns a conflict. Amazon's first-party engineering guidance similarly recommends a caller-provided request ID and an ACID boundary that includes both the ID and the mutation; a reused ID with different intent is rejected.

Store the responsibilities separately:

• payments: order reference, amount, currency, aggregate state, authorized/captured/refunded totals, and version;
• payment_operations: type, operation ID, requested amount, state, provider, provider reference, attempts, and unknown

reason;

• provider_events: provider event ID, signature result, received time, encrypted payload reference, and processing state;
• journal_entries: journal ID, account, debit or credit, amount, currency, and operation reference;
• outbox_events: a business event committed locally and awaiting publication.

The internal operation ID can become the provider idempotency key. Adyen documents safe retry with the same key after a payment timeout, along with key scope, retention, and cross-region limits. The system therefore models the actual provider contract instead of treating a UUID as a permanent, global guarantee.

Step 3: Model payment and operation as two state machines

The payment aggregate is the order- and user-facing lifecycle:

CREATED -> REQUIRES_ACTION -> AUTHORIZED -> CAPTURED
       \-> FAILED          \-> CANCELED
CAPTURED -> PARTIALLY_REFUNDED -> REFUNDED

Each authorize, capture, cancel, or refund has an independent operation lifecycle:

PENDING -> SUCCEEDED | FAILED | UNKNOWN
UNKNOWN -> SUCCEEDED | FAILED       (after query, webhook, or reconciliation)

FAILED means authoritative evidence says this operation will not later succeed. Network timeout, 5xx, or a lost response is UNKNOWN. The aggregate accepts only valid monotonic transitions. A webhook reporting an old state is retained for audit but cannot move the aggregate backwards. When a webhook and active query race, a row version or conditional update commits the transition only once. Stripe documents a payment intent as a resource spanning creation through checkout, including extra authentication, and a manual-capture payment as becoming capturable before capture. That supports a long-lived payment resource rather than equating payment with one HTTP response.

Order authorization is decoupled from inventory confirmation. Inventory success triggers capture, while failure triggers cancel. Those operations can race, so a conditional database update allows only one to claim the current AUTHORIZED version. A webhook may arrive before the provider call returns. Both the synchronous response and webhook must enter the same state-application function rather than implementing two transition paths.

Step 4: Accept external non-atomicity and make the workflow converge

A local transaction commits operation state with the outbox. The relay publishes at least once, and a worker claims by operation ID. The worker reuses that ID as the provider idempotency key and applies bounded connection, request, and total deadlines. A definite result saves the provider reference and a response digest. A lost response marks UNKNOWN and schedules status lookup. It never creates a new operation or blindly switches provider.

There are three important crash windows:

1. A crash before the local transaction commits leaves no visible operation, so the caller retries with the same key.
2. A lost publication acknowledgement after the outbox commit makes the relay publish again; the worker claims the same

operation.

1. Provider success with a lost response uses the same provider key, provider-reference lookup, or webhook to converge.

This supplies a retryable and auditable expression of one intent, not a transaction across companies. If the result is still unknown after the provider's idempotency retention window, automated retries stop. Provider reports and an operator must establish the outcome; elapsed time alone does not prove failure.

Step 5: Receive asynchronous webhooks securely and tolerate reordering

The endpoint retains the raw request body and verifies its signature and timestamp window with current and rotation-period old secrets. It rejects invalid signatures. A unique (provider, provider_event_id) receipt is persisted, after which the endpoint quickly returns 2xx and queues asynchronous processing. Logs contain event IDs, provider references, and reason codes, not complete sensitive payloads or secrets.

Webhooks can be duplicated, delayed, and reordered. Stripe explicitly documents automatic live-mode retries and no event ordering guarantee. A handler cannot assume authorization always arrives before capture. It can retrieve the provider's current resource through the event's object reference, or map the external fact to an allowed local transition. Every event deduplicates through the same operation ID or provider reference. A posted capture cannot be posted twice, and an old authorization cannot demote CAPTURED to AUTHORIZED.

The browser only polls or subscribes to local payment state. A “success” parameter on a return URL cannot fulfill an order, and the client cannot submit CAPTURED. After authoritative capture success and the ledger commit, the payment service publishes PaymentCaptured through the outbox. The order service fulfills idempotently by event ID.

Step 6: Express money facts with an append-only ledger

Payment state is an operational view; the ledger is the money audit record. This prompt simplifies accounting to two accounts: processor receivable is an asset and merchant payable is a liability. A CNY 100.00 capture, where the minor unit is the fen, posts:

journal capture-<operation_id>, CNY
  debit   processor_receivable  10000
  credit  merchant_payable      10000

A CNY 30.00 refund adds a reversing journal:

journal refund-<operation_id>, CNY
  debit   merchant_payable       3000
  credit  processor_receivable   3000

Every journal has at least two entries and equal debit and credit totals per currency. Its journal_id and business operation reference are unique. The transaction that advances authoritative state to CAPTURED or a successful refund also inserts the journal and outbox event. If fees, chargebacks, or merchant payouts enter scope, add explicit accounts and new entries; never rewrite old entries. Balances are derived from entries or accelerated by a rebuildable projection. The projection cannot become the money source of truth.

Concurrent refunds first claim capacity conditionally on the payment row. A new operation and increase in the in-flight amount are allowed only when captured_amount - refunded_amount - pending_refund_amount is sufficient. Success moves the amount from pending to refunded, definite failure releases it, and unknown retains the reservation. That prevents another refund from exceeding the limit. A provider might enforce its own refund ceiling, but the local invariant cannot depend on that external backstop.

Step 7: Reconcile silent discrepancies and repair them safely

The real-time path cannot prove that nothing was ever missed. The first reconciliation layer resolves UNKNOWN operations by provider reference or idempotency key and compares amount, currency, operation type, and terminal state. The second loads immutable provider transaction or settlement reports daily and matches payment, operation, journal, and order references. The third matches provider settlement batches to actual bank deposits and distinguishes unsettled amounts, fees, refunds, and chargebacks.

Classify discrepancies as external-only, internal-success/external-missing, wrong amount or currency, stale state, duplicate reference, unbalanced ledger, or missing settlement item. A high-risk discrepancy blocks release of the affected merchant balance and alerts. The repairer is idempotent: importing an already-confirmed provider operation adds a new journal with an audit reason, while an accounting correction uses a compensating journal rather than UPDATE on history. Stripe's reporting documentation describes balance transactions as immutable, with a new refund transaction negating the original fact, and separately reconciles payments, payout batches, and bank receipts.

Metrics separate API success and p99, payment-state distribution, count and age of UNKNOWN operations, provider errors and throttling, webhook signature failures/duplicates/delay, authorization-to-capture time, reserved refund capacity, outbox and queue backlog, rejected unbalanced journals, discrepancy count, and oldest unresolved discrepancy. Business reporting keeps authorization, capture, refund, and settlement rates distinct. “Request accepted” must not count as “money received.”

Step 8: Minimize the sensitive surface and inject failures

The provider's tokenization component collects card data directly. The backend stores only an irreversible provider token and approved display fields such as brand and last four digits. Client secrets never enter URLs or logs. Provider keys are kept and rotated in a controlled secret system. Webhook secrets are distinct, and sandbox and production are isolated. Viewing payments, initiating refunds, reading the ledger, and performing manual repairs use separate permissions. Every high-risk action records the operator and reason.

PCI SSC explicitly prohibits retaining card verification codes, PINs, and PIN blocks after authorization, even when encrypted. Hosted collection narrows this prompt's exposure, but a formal assessment still determines compliance. An interview answer cannot claim that “using a token removes PCI.”

Acceptance tests cover a client retrying the same key and changing the amount under that key; provider success with a lost response; webhook arrival before API response; duplicate, reordered, and delayed webhooks; duplicate outbox publication; authorization racing cancellation; two refunds competing for the same capacity; a full refund after a partial refund; provider idempotency expiry; webhook-secret rotation; rejection of a one-sided journal; an external-only transaction found by reconciliation; repeated repair execution; and catch-up after prolonged queue or provider outage. Every scenario asserts payment state, operation state, journal count, order side effects, and alerts.

High-Quality Sample Answer

“I would model payment as a long-lived business resource rather than one HTTP call. One order has a stable payment_id. The aggregate stores amount, currency, and authorization/capture/refund state. Each authorize, capture, cancel, and refund uses a separate operation ID and PENDING, SUCCEEDED, FAILED, or UNKNOWN. An external timeout becomes UNKNOWN; it cannot immediately fail or switch provider.

Payment creation and every money operation require a caller idempotency key. Merchant, operation type, and key are unique. The first request atomically stores its digest, payment or operation, and outbox; the same request replays its result, while a different amount or currency under the key conflicts. A worker uses the internal operation ID as the provider key. The outbox, queue, and worker are all at least once, and the same operation applies one authoritative transition.

The provider's synchronous response, a signature-verified webhook, and active lookup enter the same state-application function. The webhook verifies the raw payload, deduplicates its event ID, persists before returning 2xx, and tolerates duplicates and reordering. An old event cannot move payment backward. The browser only displays local state and cannot trigger fulfillment. Only authoritative capture success committed with a balanced journal and business outbox allows the order service to fulfill idempotently.

A capture journal debits processor receivable and credits merchant payable. A refund adds a reversing journal; history is immutable. Refund creation conditionally reserves available refundable value, so confirmed plus in-flight refunds never exceed capture. Money uses integer minor units, currency is immutable after the first attempt, and each currency balances independently.

Recovery has three layers: unknown operations reuse the provider key or query status, daily provider reports reconcile payments, operations, and journals, and settlement batches match bank deposits. Discrepancies are quarantined and alerted; repair only appends an idempotent import or compensating journal. A hosted provider component collects the card, so the backend stores neither the card number nor security code. Lost responses, duplicate or reordered webhooks, concurrent capture and refunds, provider outage, an unbalanced journal, and an external-only reconciliation item then prove that each external result converges to one auditable money fact.”

Common Mistakes

• Treat the browser success page as payment success → the redirect is forgeable, and payment may still await

authentication or asynchronous confirmation → **Only authoritative provider evidence accepted by the local state machine triggers fulfillment.**

• Use one paid field for the whole lifecycle → it cannot express extra authentication, authorization, capture,

partial refund, or unknown outcome → Separate the payment aggregate from operation attempts.

• Retry with a new ID or another provider after timeout → the first attempt may have charged, creating a double charge

→ Reuse the operation and provider key, then query or reconcile unknown results.

• Compare only the idempotency key, not parameters → a caller reusing a key with a changed amount gets the wrong

business result → Persist a normalized request digest and conflict on changed intent.

• Depend on webhook order → the provider can retry, delay, and reorder events → **Deduplicate and retrieve current

state or apply only legal monotonic transitions.**

• Treat the payment-table balance as a ledger → in-place updates lose the reason for money changes and cannot be

reconciled independently → Append balanced journals; keep balance as a rebuildable projection.

• Read then write refundable balance concurrently → two callers can both see capacity and exceed capture → **Reserve

capacity with a transactional conditional update tied uniquely to the operation.**

• Monitor only API 200 responses → accepted, authorized, captured, and settled have different meanings → **Measure

each state, unknown age, and reconciliation discrepancies.**

• Claim tokenization automatically removes PCI responsibility → pages, logs, scripts, and operational processes can

remain in scope → Minimize card data and have compliance confirm the actual boundary.

• Edit historical entries to repair a discrepancy → the audit chain breaks and historical reports cannot replay →

Append an explained import or compensating journal.

Follow-Up Questions and Responses

Follow-up 1: The provider captured, but both the synchronous response and webhook were lost. What happens?

The operation remains UNKNOWN, retaining the related authorization or refund reservation and preventing another operation for the same intent. First retry a supported lookup or call with the original provider idempotency key, then actively query by provider reference. If it remains indeterminate, wait for transaction-report reconciliation. Confirmed success enters the same state-application function and posts the journal and outbox; only confirmed failure releases capacity. After provider key retention expires without evidence, send the case to operators rather than inferring failure from elapsed time.

Follow-up 2: Two CNY 60 refunds concurrently target a CNY 100 capture. How do you prevent over-refund?

Use a conditional update on the payment or dedicated refundable-balance row. It increases pending_refund_amount by CNY 60 and creates the operation only if at least CNY 60 remains. Both transactions contend on one version, so one succeeds and the other rereads insufficient capacity. An unknown refund retains its reservation, a definite failure releases it, and a success moves pending to refunded. Provider enforcement is only the second line of defense.

Follow-up 3: CAPTURED arrives by webhook before AUTHORIZED. How is it processed?

Persist and deduplicate both receipts. If the CAPTURED signature, amount, currency, and provider reference match, apply the capture transition and journal. The later AUTHORIZED event is an older fact. The state machine rejects rollback and updates only webhook audit and delay metrics. If the payload lacks sufficient version evidence, retrieve the provider's current payment resource instead of overwriting by arrival order.

Follow-up 4: Why have both a payment table and a ledger?

The payment table is a workflow aggregate suited to answering whether capture, cancellation, or refund is allowed. The append-only ledger is a money record suited to explaining how a balance formed and which business event caused each change. Payment can move from CAPTURED to PARTIALLY_REFUNDED, while the ledger retains the original capture and each independent balanced refund journal. A unique operation reference joins them, and an authoritative transition commits both in one local transaction. Neither responsibility replaces the other.

Follow-up 5: How do you prove a reconciliation repairer cannot post twice?

Give each external report row a stable source key such as provider, report type, and transaction reference. Bind the repair to the discrepancy ID and add a unique (source_key, repair_type) constraint. The first transaction appends the journal, marks the discrepancy, and writes the outbox. A crash and rerun finds the same repair and replays its result. Kill the process before commit, after commit, and after event publication; journal count, balance, and downstream event count must never increase a second time.

Follow-up 6: If a second provider is added later, when is failover allowed?

Create an operation at the second provider only when the first explicitly says the operation was never created or definitively failed, and the local operation has no money fact. Connection timeout, 5xx, and unknown do not satisfy that condition. Audit routing choice, provider capability, amount, currency, and reason. If both outcomes might exist, freeze fulfillment and balance release while query and reconciliation eliminate a double charge. An automatic refund cannot hide an unknown outcome.