Backend Interview: How Do You Keep a Database and Cache Consistent?

Prompt and Applicable Context

PostgreSQL is the source of truth for product data. Redis stores a display snapshot keyed by product_id. The system handles about 20,000 reads and 500 writes per second across roughly 2,000 active products. Names, images, and display copy may be stale for at most 5 seconds. Inventory deductions, checkout prices, permissions, and balances are outside that cache contract because they determine business correctness.

Design the cache-aside read, update, and invalidation paths. Cover these cases:

• the database commits, but the process crashes before deleting the cache entry;
• a reader obtains an old database version and completes its fill only after a writer invalidates the cache;
• a fill reads lagging data from an asynchronous database replica;
• invalidation events are duplicated, reordered, or backlogged;
• Redis is temporarily unavailable;
• the product team wants “at most 5 seconds stale” to be a monitored and tested contract.

The throughput, active-key count, and 5-second budget are interview assumptions, not universal settings. Current public 2026 backend and senior caching interview material explicitly covers cache-aside, invalidation after writes, and cache consistency. The central skill is the backend protocol and failure semantics after a database commit, so the category is backend. This is distinct from preventing a cache stampede when a hot key expires: stampede control limits concurrent source reads, while this problem asks when an old value can survive a write, why it survives, and how long it may remain.

What the Interviewer Evaluates

First, can the candidate define the consistency target before choosing a pattern? “The database and cache are always consistent” does not specify which read results are legal. A strong answer separates 5-second bounded staleness for display data, read-your-writes for the caller that performed an update, and authoritative reads for inventory or permissions. Those contracts require different paths.

Second, can they explain the write order? A common cache-aside write path commits the database and then deletes the cache entry. Deleting first creates a clear race: a reader misses between the two operations, fetches the old database value, and puts it back in the cache. Database-first is still not an atomic dual write. There is a brief commit-to-delete window, and a failed delete can leave an old entry until its TTL.

Third, can they draw the late stale-fill timeline? A reader may fetch version 41 before a database update. A writer then commits version 42 and deletes the cache, after which the old reader puts version 41 back. Adding a version only to the cached value is not always enough. After deletion there is no cached version to compare with, so a rule such as “write when the source version is at least the cached version” accepts 41. The design needs a version fence that survives value deletion, a fill lease that writes invalidate, or another authoritative version check before the fill.

Fourth, can they distinguish reliable invalidation from a finite time bound? A transactional outbox or change data capture closes the gap in which the database commits but no invalidation message is durably recorded. At-least-once delivery and idempotent deletion tolerate duplicates. Retries establish eventual processing; they do not prove completion within 5 seconds. Bounded staleness additionally needs a hard TTL, an invalidation-lag gate, or a rule that bypasses the cache once the pipeline exceeds its budget.

Fifth, can they handle the source of truth, database replicas, and operational verification? A successful database commit creates the new fact. Filling from a lagging replica can reintroduce an old value after a correct invalidation. A strong answer constrains the fill source, observes event and cache versions, and tests controlled concurrency timelines and failures instead of reporting only cache hit rate.

Questions to Clarify Before Answering

• What is the consistency contract? If up to 5 seconds of staleness is allowed, asynchronous invalidation plus a hard deadline can work. If read-your-writes is required, subsequent requests from the writer must bypass the cache briefly or carry a minimum version. If no stale read is legal, read the authoritative store or use a storage path with the required consistency contract.
• Which fields authorize irreversible actions? A display name or image may be stale. Inventory validation, discount eligibility, permissions, balances, and charged prices must not treat a cached snapshot as authority. If those fields share one object, split the read contracts or re-read authoritative state during the critical action.
• When does the 5-second clock start? In this prompt it starts at the database commit. If it starts when the value enters Redis, a replica that is already 4 seconds behind can be cached for 5 more seconds, making the real data age nearly 9 seconds.
• Does one service control every write path? Batch jobs, admin tools, and other services can bypass an API-level DEL. Put an invalidation record in the same database transaction or capture every supported change from the database log.
• Does a miss fill from the primary or a replica? Establish the replica-lag bound and whether session-level read-your-writes exists. If the replica cannot be proven current within the budget, the first post-invalidation fill should use the primary or require a result at least as new as the caller's minimum version.
• How much fill traffic can the database absorb? With a 5-second TTL, 2,000 uniformly expiring active keys that are all read produce about 2000 / 5 = 400 fills per second on average. Access distribution, jitter, and request coalescing change the actual value. If the database lacks that budget, shortening the TTL does not magically satisfy consistency.
• When Redis is unavailable, is freshness or availability more important? Display data can degrade within an explicit stale-value bound. Authoritative fields must use a protected source path. If the database cannot absorb all misses, apply rate limits, bulkheads, and explicit failures rather than an unbounded fallback.

30-Second Answer Framework

“I first define contracts by data type. Product-display data may be stale for at most 5 seconds from the database commit, while inventory and permissions always use an authoritative path. A read checks Redis, coalesces same-key misses, fetches a versioned row from a source that satisfies the freshness requirement, and conditionally fills the cache. A write updates the business row and inserts an outbox record containing the row version in one PostgreSQL transaction. After commit, it attempts a fast cache delete, while the outbox or CDC path provides retryable repair.

I commit the database before deleting the cache. To stop an old read from filling after that deletion, a miss captures a fill generation; invalidation advances a version fence and deletes the value; the fill succeeds only if the generation is unchanged and the source version meets the fence. Eventual retries do not prove a 5-second bound, so the cache has a hard TTL within the budget, and reads bypass it as invalidation lag approaches the limit. A post-invalidation fill uses the primary when replica lag cannot meet the contract. I verify the design with commit-then-crash, late-fill, duplicate and reordered event, and replica-lag fault tests, asserting stale age, monotonic versions, and database load.”

Step-by-Step Deep Dive

Step 1: Turn consistency into a read-result contract

Split the data by business consequence before selecting a cache pattern:

This classification drives the rest of the design. A cache accelerates a rebuildable copy. It must not authorize an inventory deduction using a value that can be delayed, evicted, or lost. A 99.9% hit rate says nothing about whether the other 0.1% produces overselling or unauthorized access.

Each cache entry needs at least a source version and timing information. The following is a pseudostructure, not an executable type in a particular language:

ProductCacheEntry {
  value
  source_version
  source_committed_at
  cached_at
}

source_committed_at measures real stale age; cached_at says only when Redis received the value. The source version can be a monotonically increasing row version, a commit sequence, or another domain version with a defined order. A wall-clock timestamp alone is a poor ordering proof if values can tie or clocks can drift.

Step 2: Establish the basic cache-aside paths

The read path returns a cache hit only when it satisfies both the caller's minimum version and the stale-age budget. On a miss, coalesce fills for the same product_id so that 20,000 reads do not reach the database together. Fetch a versioned row, then attempt a conditional Redis write. The following is flow pseudocode:

read(product_id, min_version = none):
  entry = cache.get(product_id)
  if entry satisfies age_budget and min_version:
    return entry.value

  return singleflight(product_id):
    recheck cache
    row = read_authoritative_version(product_id)
    conditional_fill(product_id, row)
    return row.value

The write path updates the business row and inserts an outbox record in the same PostgreSQL transaction. Only a successful COMMIT makes the business change visible to other transactions and durable. The application then performs a fast invalidation attempt. A separate relay or CDC consumer processes the durable invalidation event for retry and repair. The write-path pseudocode is:

transaction:
  row = update product and increment source_version
  insert outbox(product_id, source_version, committed_at)
commit

best_effort_invalidate(product_id, source_version)
return committed source_version

The direct invalidation reduces the common-case window. The outbox closes the message-loss gap when the process crashes after commit. If both paths handle the same change, deletion must be idempotent: receiving the same source version again cannot cause a new business side effect.

Step 3: Analyze the three race windows explicitly

Window 1: between database commit and cache deletion.

W: COMMIT version 42
R: read cached version 41
W: DEL cache key

The reader briefly sees version 41, which is legal under bounded staleness. If no stale result is acceptable, waiting for a cache operation before returning the write response still does not resolve every network ambiguity. A clearer contract sends the critical read to the source of truth or lets the caller require min_version=42.

Window 2: the database commits but invalidation does not run.

W: COMMIT version 42 plus outbox record
W: process crashes before DEL
R: cache still contains version 41
relay: retries invalidation for version 42

An in-memory task created after COMMIT can be lost. The transactional outbox commits the business change and the obligation to invalidate together. The relay may redeliver, while the consumer deletes by key and records the handled version. If the relay stalls, the hard TTL or freshness gate enforces the 5-second boundary.

Window 3: an old read fills after invalidation.

R: cache miss; captures generation 7
R: reads database version 41
W: commits version 42
W: advances generation to 8 and deletes cache value
R: tries to fill version 41 with generation 7; rejected

This is the frequently missed race. If the design deletes only the value, then version 41 >= no version passes and resurrects stale data. One solution separates the value from its fence. Invalidation advances a short-lived generation or minimum-source-version key. A Redis script atomically checks that the fill generation still matches the generation captured at miss time and that the source version meets the minimum before writing the value. In Redis Cluster, the value and fence keys must use the same hash slot so that the script can access both atomically. Another implementation issues a miss lease that a database update invalidates. Re-reading the primary version before filling can help, but adds a database read and still needs a defined atomic boundary between the check and the cache write.

Retain the version fence long enough to cover the maximum fill duration, retries, and process or network pauses. Removing it too early reopens the late-fill window. The fence protects cache-write ordering; it does not replace database concurrency control or tell an ordinary reader that the database contains a newer version whose invalidation has not arrived yet.

Step 4: Make invalidation recoverable without overstating delivery

The outbox row and business update are written in one transaction. A relay sends the invalidation to a durable channel. The consumer can track the highest observed version for each product_id and apply these rules:

1. An out-of-order event below the highest observed version is acknowledged idempotently.
2. A new version advances the minimum-version fence and deletes the cached value.
3. A failed cache command is retried; exhausted work enters a visible quarantine queue.
4. A reconciliation job compares database versions, outbox progress, and sampled cache versions.

At-least-once delivery works well with deletion because a duplicate DEL has no additional business meaning. Do not claim that retries create exactly-once execution. A consumer can delete in Redis, crash before acknowledging the message, and delete again. Safe repetition and detectable gaps are the useful properties.

Observe the interval from database commit to successful invalidation, not only broker queue age. Useful metrics include invalidation_lag_seconds, invalidation failures and retries, oldest quarantine age, cache-to-source version lag, over-budget cache bypasses, database fill QPS, and the number of same-key calls shared through singleflight.

Step 5: Prove the 5-second bound with a TTL and a gate

A reliable event eventually arrives, but “eventually” has no time unit. A contract of at most 5 seconds stale from commit needs an independent finite-time defense:

• Set the physical cache TTL below 5 seconds after reserving margin for clocks, scheduling, and detection, and compute stale age from the authoritative version's commit time.
• Alternatively, as invalidation lag approaches the budget, stop trusting the cache globally or by affected partition and read a source that meets the freshness requirement.
• If neither is possible, the contract is eventual consistency and must not continue to claim a 5-second bound.

The TTL is a boundary backstop, not the primary invalidation mechanism. Add jitter so that 2,000 keys do not expire at once, and coalesce misses for each key. If all 2,000 active keys are read at least once per 5-second interval, uniformly distributed expiration produces about 400 fills per second on average. That estimate is not a capacity guarantee. Popularity skew, batch expiry, Redis failure, and slow queries can create peaks, so test against safe database QPS and concurrency bulkheads.

If a 5-second TTL exceeds database capacity, there are three honest choices: add safe fill capacity, reduce the active data that needs this contract, or relax the stale budget. Quietly increasing the TTL violates the prompt.

Step 6: Handle replica lag and read-your-writes

Even after a correct deletion, a lagging replica can refill an old version. For a miss with a hard 5-second contract, use this order of preference:

1. Read the primary for the first fill after invalidation.
2. Use a replica only when an observable replay position has reached the required commit position.
3. Return source_version from a write and let subsequent reads carry min_version; route to the primary when the cache or replica is behind.
4. Open a freshness circuit breaker when replica or invalidation lag exceeds the budget, and stop returning unqualified cached values.

A 5-second TTL does not prove 5-second data age if the fill came from a replica already 4 seconds behind. Stale age starts at the source-of-truth commit, so replica, event-channel, and cache delays all count.

Step 7: Compare alternatives and their boundaries

Synchronously update the cache. Writing Redis immediately after the database may improve read-your-writes, but two independent systems still have partial failures. The database can commit while the cache update fails, and concurrent database writes can reach the cache in a different order. This path still needs source-version conditions and repair; calling it write-through does not make the two writes an atomic transaction.

Delete first, then update the database. This is simple but allows a reader to refill old data before the database commit. A delayed second delete reduces the probability of a particular timing, but a fixed delay cannot cover unbounded replica lag, process pauses, or network faults. The process can also crash before the second delete. It may be an auxiliary measure, but it does not prove bounded staleness by itself.

Use only a short TTL. This can be the simplest adequate choice when writes are rare, the stale budget is generous, and the database can absorb fills. The cost is that every write may be followed by stale reads for the full TTL, while synchronized expiry can amplify database load.

Read everything from the database. For low-throughput or correctness-critical data, this is often the clearest solution. A cache is optional. When consistency coordination costs more than the database reads it saves, removing the cache is reasonable.

Step 8: Test invariants and failure paths

Do more than manually refresh a page after an update. Use barriers to reproduce the late stale fill. Pause a reader after it obtains version 41. Let a writer commit version 42, advance the fence, and delete the value. Release the old reader and assert that its conditional fill fails. Then test these cases:

• terminate the writer immediately after PostgreSQL commits and verify that the outbox eventually invalidates;
• duplicate and reorder one invalidation event and verify that the highest version never regresses;
• fail the Redis delete and verify retries, quarantine, and the TTL backstop;
• pause replica replay and verify that fills use the primary or reject an unqualified result;
• delay invalidation consumption beyond its gate and verify that reads bypass the cache;
• bring 2,000 keys near expiry together and verify jitter, singleflight, and database bulkheads;
• make Redis unavailable and verify budgeted degradation for display reads while authoritative decisions keep their required path.

Core invariants are: a returned cache version is never below the caller's min_version; an invalidated fill generation cannot write; an authoritative business action never trusts the display cache; stale age does not exceed 5 seconds; and database fills remain within tested safe QPS and concurrency.

High-Quality Sample Answer

“I would not begin by promising that PostgreSQL and Redis match at every instant. I would split the read contracts. Product names and images may be stale for at most 5 seconds from the PostgreSQL commit. Inventory deductions, permissions, balances, and checkout decisions continue to use authoritative state. When a writer needs read-your-writes, the write API returns the source version. A subsequent read supplies that minimum version and uses the primary when Redis is older.

The base pattern is cache-aside. A read returns a hit only when its age and version qualify. A miss uses singleflight by product_id, fetches a monotonically versioned row, and performs a conditional fill. One PostgreSQL transaction updates the product, increments its version, and inserts an outbox row. After commit, the application tries to delete Redis immediately. A relay or CDC consumer retries from the durable event, so a crash after commit does not permanently lose the invalidation obligation.

The order is database first, cache delete second. Deleting first lets a reader restore old data before the database commits. Even with the right order, a late fill remains: a reader gets version 41, a writer commits 42 and deletes, then the old reader sets 41. A version inside the cached value alone is insufficient because there is no 42 left after deletion. I have the miss capture a generation. Invalidation advances the generation and minimum version before deleting the value. An atomic script rejects the fill unless the generation is unchanged and the source version meets the fence. If a replica cannot prove it has replayed the required commit, the post-invalidation fill reads the primary.

The outbox proves recoverable eventual invalidation, not a 5-second deadline. I set a hard TTL within the budget, with operational margin, and measure commit-to-invalidation lag. As that lag approaches the budget, reads bypass Redis. If all 2,000 active keys are accessed every 5 seconds, uniform fills average about 400 per second, so I also use TTL jitter, per-key request coalescing, and a database concurrency bulkhead.

For verification, I control thread ordering and inject commit-then-crash, late-reader, duplicate and reordered event, failed Redis delete, and replica-lag faults. Acceptance means an old generation cannot fill, source versions do not regress, stale age stays within 5 seconds, authoritative actions bypass the display cache, and database load remains within the measured budget.”

Common Mistakes

• Claiming “strong consistency between database and cache” → The answer does not define legal reads, failure deadlines, or a transaction boundary across the two systems → State separate contracts for bounded staleness, read-your-writes, and authoritative reads.
• Deleting the cache before updating the database → A miss between those operations reads and restores the old database value → Commit the database first, then invalidate, with durable repair.
• Publishing only an in-memory message after commit → A crash before publication permanently loses the invalidation → Write an outbox in the same transaction or capture database-log changes.
• Assuming DEL ends every race → An older read can complete its set after the deletion → Reject late stale values with a fill lease or a version fence that survives value deletion.
• Putting a version only inside the cache value → An empty cache has no newer version to compare with and can accept an old value → Keep the minimum version or generation in a separate fence and check the fill atomically.
• Treating duplicate consumption as an error → A consumer crash before acknowledgement naturally creates redelivery → Make deletion and highest-version advancement idempotent under at-least-once delivery.
• Using an outbox to promise 5 seconds → Retriability proves eventual handling, not a finite delay → Add a hard TTL or an over-budget bypass gate and measure end-to-end commit-to-invalidation lag.
• Filling from any read replica → Replica lag can repopulate old data after correct invalidation → Check replay position, use the primary, or require a minimum source version.
• Caching all fields in one display snapshot → The display-data stale allowance leaks into inventory, permissions, and checkout → Split data contracts and re-read authoritative state for irreversible actions.
• Sending all traffic to the database when Redis fails → 20,000 reads per second may overwhelm the source of truth → Protect it with bulkheads, rate limits, explicit degradation, and controlled recovery.
• Using a fixed delayed double delete → A sleep cannot cover unbounded lag and the second delete can also be lost → Treat it as a probability optimization while retaining durable invalidation, a fence, and a finite-time backstop.

Follow-Up Questions and Responses

Follow-up 1: What if product prices also require read-your-writes?

Return the committed source_version from the write API. A subsequent read in that session sends min_version; if Redis is older, read the primary and conditionally cache only the new version. If the writer merely needs to display the result, return the committed row directly and bypass the cache briefly. Checkout must still revalidate the price in the authoritative transaction; read-your-writes is not payment authorization.

Follow-up 2: Why can the version fence not live only in the cached value?

Invalidation deletes that value, so an ordinary comparison can no longer see version 42. A request that previously read version 41 can then treat 41 as a valid value for an empty key. A separate fence or miss lease survives the value deletion and records either that version 42 exists or that the old fill permission has been revoked.

Follow-up 3: Can a transactional outbox send duplicates?

Yes. The relay can crash after the downstream system accepts an event but before it marks the outbox row complete. The consumer handles (product_id, source_version) idempotently: an old version does not advance state, a new version advances the maximum and deletes the value, and duplicate deletion is safe. The useful guarantee is no silent invalidation loss plus reconciliation, not exactly-once execution.

Follow-up 4: Can the TTL be 30 minutes if CDC handles invalidation?

That can be a valid trade-off when the business requires only eventual consistency and accepts a prolonged invalidation outage. This prompt promises at most 5 seconds stale. A 30-minute TTL violates that bound when CDC stalls unless the system automatically bypasses the cache as invalidation lag approaches 5 seconds. Otherwise, keep a hard deadline within the budget.

Follow-up 5: Replica lag is normally only tens of milliseconds. Why use the primary?

“Normally” is not an upper bound. Deployments, network partitions, long transactions, and recovery can increase lag. A replica remains usable when its replay position is proven to include the required commit or the read carries a minimum version. If that cannot be proven, route the critical miss to the primary. The choice follows the 5-second contract and the database capacity budget.

Follow-up 6: Why not update PostgreSQL and Redis while holding one distributed lock?

A lock constrains only participants that follow that protocol. It cannot make two systems commit atomically, nor eliminate holder crashes, lock expiry, network partitions, or external database writers. The database transaction establishes the fact, and durable invalidation repairs the second system. A lock may reduce same-key concurrency, but it is not a consistency proof.

Follow-up 7: How do you keep the 5-second bound when Redis is fully unavailable?

Display reads bypass Redis, but every source read passes through database bulkheads and rate limits. If capacity is insufficient, return an explicit degraded result or failure. A process-local old copy may serve briefly while still inside the 5-second budget; after that it is disqualified. Recover with rate-limited warming while draining invalidations so that a cold cache does not overwhelm the database again.

Follow-up 8: How do you prove production does not contain long-lived stale values?

For sampled keys, read the database source version and cached version together, and record both version distance and age from source commit. Reconcile database changes, outbox progress, and consumer high-water marks. Alert on end-to-end invalidation lag, oldest quarantine work, and over-budget bypasses. Regularly inject commit-then-crash, Redis-command failure, and replica-pause faults to show that the gate and TTL still protect the invariants under real failure.