Data Engineering Interview: How Do You Debug Wrong Dashboard Numbers?

Prompt and Applicable Context

After a data-pipeline release, the daily revenue dashboard is 8% higher than the payment processor's reconciliation for the same business day. Event ingestion is at least once, refunds can arrive late, and finance closes the books in two hours. Explain how you would confirm impact, locate the root cause, stop the bad data from spreading, repair the affected data, and prove that the dashboard is trustworthy again.

The 8%, two-hour deadline, at-least-once delivery, and release time are interview assumptions, not industry benchmarks. The primary path assumes this lineage: payment-event source, raw layer, staging layer, revenue fact table, semantic layer, and BI cache. The payment processor is a candidate comparator, not automatic ground truth. If it groups by settlement date while the dashboard groups by payment-event date, both outputs can be correct and still differ.

Public 2026 data-engineering interview materials directly include the “dashboard shows wrong numbers” prompt and list data quality, lineage, backfills, SLAs, incident response, and ownership as preparation topics. The category is data because the core skills are metric semantics, data lineage, quality assertions, reconciliation, and safe backfills. The question does not ask for a full cross-component platform design.

What the Interviewer Evaluates

The first signal is whether the candidate distinguishes “different numbers” from “bad data.” Rerunning a job immediately can write the same defect again. A strong answer first fixes the business day, time zone, currency, order states, and whether revenue means authorization, capture, settlement, or net amount after refunds. Only then can the candidate decide whether the 8% gap is a quality incident or a semantic mismatch.

The second signal is whether lineage is used to find the first bad boundary. Editing the final dashboard SQL cannot explain how the defect entered the system. A useful diagnosis compares counts, amounts, and states for the same business keys at the source, raw layer, staging layer, fact table, semantic layer, and cache. The target is the transition where the upstream side is still correct and the downstream side first becomes wrong.

The third signal is incident control. With finance about to close, the candidate must reduce recovery time without creating a second corruption through an unverified fix. That means marking the dashboard unsafe for close, pausing exports or reverse syncs that use the bad data, preserving immutable raw inputs, and backfilling into a shadow table or versioned partition.

Finally, the interviewer wants a closed evidence loop. A successful job, equal row counts, or a dashboard that “looks normal” does not prove recovery. A strong answer combines business-semantic reconciliation, key and join-cardinality checks, diffs for affected slices, audits of critical records, and finance sign-off before restoring consumption.

Questions to Clarify Before Answering

• Do both sides define revenue identically? Align authorization, capture, settlement, refund, chargeback, tax, fee,

and canceled-order treatment. If definitions differ, first construct a comparable view instead of calling the expected gap an incident.

• Which clock and time zone define the business day? UTC, merchant-local time, and processor settlement date can cut

different boundaries. The answer changes the affected partitions and repair plan.

• Is 8% an amount gap, a count gap, or a slice-specific gap? Normal counts with excess value suggest duplicated

high-value events, foreign exchange, or join fanout. Excess counts make replay and deduplication higher-priority checks.

• What changed, and when did it take effect? Link code version, job run ID, input and output datasets, and the first

anomalous time. Release timing makes a useful hypothesis, not proof that justifies an immediate rollback.

• Are raw events immutable, with a stable event_id? If so, the team may rebuild a business day idempotently. If

not, recovery needs an upstream ledger or snapshot and an explicit boundary on what cannot be reconstructed exactly.

• When do refunds and exchange rates mature? If the dashboard promises a near-real-time estimate while reconciliation

includes complete refunds only on T+1, show preliminary and final values separately and define a correction window.

• Which consumers depend on the table? Finance exports, executive dashboards, alerts, machine-learning features, and

reverse ETL carry different risks, so containment should follow business impact.

• What is the current trusted version? A validated pre-release snapshot or partition can temporarily serve a

timestamped known-good view. Without one, show a degraded state instead of silently serving stale numbers.

30-Second Answer Framework

“I would first freeze this dashboard for financial close and pause affected downstream exports while preserving raw events. Then I would align the revenue definition, business day, currency, and refund window to confirm that the 8% gap is a real quality defect. Using the release metadata and lineage, I would move from the dashboard query through the semantic layer, fact table, and staging layer to raw events, comparing unique-event counts, net amounts, and key states at each boundary. The first divergence identifies the fault domain. I would rebuild the affected partitions idempotently into a shadow table using stable event IDs. After source-to-target reconciliation, join-cardinality checks, critical slices, and finance samples pass, I would atomically switch the version and refresh caches. Finally, I would put the metric definition, owner, assertions, release identity, and alerts into the data contract and runbook.”

Step-by-Step Deep Answer

Step 1: Establish facts that can decide the incident.

Record detection time, first affected business day, release version, affected dashboards, and the close deadline. Split “8% high” into at least four reproducible quantities: event count, order count, captured amount, and net amount after refunds. Slice them by currency, region, payment state, and hour. Run the dashboard query directly to bypass browser cache, then run the SQL generated by the semantic layer. If the SQL is correct but the page is wrong, the defect is in filters, cache, or presentation; do not backfill the fact table.

Write the metric as an explicit equation. For this scenario, net revenue can mean successful captured amount minus confirmed refunds and chargebacks for the same business day and currency. If fees, taxes, or foreign-exchange gains belong in revenue, add them explicitly. Do not change the definition mid-investigation to erase the gap. Map the processor's settlement report onto the same state and time semantics before treating it as a comparator.

Step 2: Contain impact and preserve evidence.

Mark the dashboard as “under data validation,” with the last trusted timestamp and next update time. Tell finance not to close from the affected partition. Pause exports, reports, and reverse syncs that would propagate the wrong amount. If pre-release partitions are already validated, isolate only the affected business day instead of taking the entire history offline.

Do not delete raw events, overwrite the current table, or immediately truncate a partition. Preserve job logs, run IDs, code versions, dataset versions, input partitions, and failed assertions. OpenLineage's Job, Run, and Dataset model shows why those identifiers belong together: knowing which run read which inputs and produced which outputs is what makes blast radius and bounded repair reproducible.

Step 3: Trace lineage to the first bad boundary.

For the same business day and business keys, build this downstream-to-upstream checklist:

Use the same business day, currency, and state set at every layer. Compare aggregates first, then anti-join the differences and sample business keys. The first boundary with a discrepancy reduces “anything in the pipeline” to one transformation or transport step.

Candidate hypotheses after a release include: an at-least-once replay that is not deduplicated by event_id; an order join that fans out against a multi-row dimension; refunds partitioned by processing time while captures use event time; a whole batch retried after only some partitions completed; an exchange-rate join matching multiple valid versions; or a new state defaulting into revenue. These are hypotheses to falsify. Give each a prediction, such as “if a dimension join fans out, amplification occurs only for currencies with duplicate dimension keys,” and test it before changing code.

Step 4: Choose the recovery mechanism.

If the defect is only cache, invalidate the relevant keys and verify the new query. If the semantic formula is wrong, version the correction and check every dashboard that consumes the metric. If the fact table is corrupt, identify the smallest affected partitions and trusted inputs, then rebuild into a shadow table or new data version:

1. Deduplicate by immutable event_id; when an event has versions, apply an explicit version or state-transition rule.
2. Associate refunds, chargebacks, and currency by business key so repeated execution returns the same result.
3. Bound the backfill and throttle warehouse load so normal incremental jobs continue safely.
4. Run structural, business, and reconciliation checks on the shadow output instead of overwriting production directly.
5. After validation, atomically switch the view or table version, refresh BI caches, and resume downstream jobs.

If raw data contains duplicates but stable keys exist, rebuild from raw. If critical events are missing and the upstream has a ledger or snapshot, re-extract from that source. If no recoverable source exists, do not claim exact repair. Give finance the confirmed scope, unexplained difference, and adjustment plan.

Step 5: Prove recovery with three classes of checks.

Structural checks cover schema, non-null, uniqueness, accepted values, and referential integrity. dbt documents unique, not_null, accepted_values, and relationships as built-in generic data tests. They catch duplicate keys, null keys, unknown states, and orphan records, but they do not replace business reconciliation.

Business checks enforce metric invariants: a refund cannot be deducted twice, an order's net amount cannot exceed its successful captured amount, and joining the fact table must not unexpectedly increase the number of order keys. Reconciliation compares source and shadow counts and amounts by business day, currency, and state, then examines the record-level difference. Equal totals are insufficient because an overcount and an omission can cancel each other.

Define recovery gates in advance: every hard assertion for the affected slices passes; every source-to-target difference is explained by semantics, the late-arrival window, or a recorded exception; sampled captures, refunds, and multi-currency orders trace end to end; and finance confirms the close definition. Observe at least one normal incremental cycle so the next run does not recreate the defect.

Step 6: Turn the failure mode into a guardrail.

A data contract should contain schema, field semantics, business day, currency, state mappings, quality thresholds, service objectives, owners, and escalation paths. Data Contract CLI documents machine-readable contracts that combine structure, semantics, quality, and service levels and can be checked in CI or against actual data.

Place each check at the earliest useful boundary: schema and primary-key checks at ingestion, join-cardinality and business invariants after transformation, and freshness, completeness, and source reconciliation before delivery. Treat freshness and correctness separately; a table delivered on time but 8% high still fails. Attach code version, run ID, and output-data version to release metadata. Shadow-run critical partitions and compare data before shifting readers.

Alerts must be actionable: name the dataset, business day, failed rule, actual value, threshold, downstream impact, and owner. A low-risk exploratory table may continue after a warning; a financial revenue table should block publication on key or reconciliation failures. Add a backfill runbook after the incident, and rehearse duplicates, late refunds, schema changes, partial writes, and join fanout to prove that recovery itself is idempotent.

High-Quality Sample Answer

“I would not rerun the pipeline first. The 8% gap may be semantic, and a rerun may amplify the same write defect. I would tell finance to stop using the affected business day for close, mark the last trusted time, pause exports from the bad partition, and preserve both raw events and current outputs for investigation.

Next I would align business day, time zone, currency, and state on both sides and determine whether revenue means captured amount or net amount after refunds. If the processor reports by settlement day while the dashboard reports by payment day, I would build a comparable view first. Once the defect is confirmed, I would split the 8% into orders, events, captures, and refunds, then segment by hour, currency, region, and state to find when and where it begins.

I would trace upward from the dashboard query through the semantic layer, fact table, staging, raw layer, and payment source. At each layer I would compare unique-event counts, net amounts, and record differences for the same business keys, looking for the first correct-to-wrong boundary and linking it to the release and job run ID. Normal fact-table order counts that grow after a semantic join suggest fanout. Duplicate raw and fact events suggest missing deduplication after an at-least-once replay. Only refunds being low suggests event-time, processing-time, or late-window handling.

I would rebuild only the affected partitions into a shadow table. Deduplication uses immutable event IDs, and refund and state-transition rules must make repeated backfills return the same output. The shadow table must pass key, non-null, state, referential-integrity, join-cardinality, and business-invariant checks. Then I would reconcile source and output by business day, currency, and state and inspect record differences. Only after finance signs off would I atomically switch versions, refresh caches, resume downstream jobs, and watch the next incremental cycle.

Finally, I would put the revenue definition, business day, currency, late-correction window, owner, and escalation path in the data contract. Ingestion, transformation, and delivery boundaries get schema, dedupe, cardinality, freshness, and source-reconciliation checks. Future releases shadow-run affected partitions and block the switch when hard assertions fail.”

Common Mistakes

• Rolling back because the timing matches a release → Correlation does not prove causation; semantics or upstream data may also have changed → Confirm with the first bad boundary and version evidence.
• Rerunning the entire DAG immediately → A non-idempotent job can duplicate writes and widen corruption → Contain first, then run a bounded backfill into shadow output.
• Treating the processor as absolute truth → Settlement day, refund windows, and state definitions may differ → Map both sides to the same business semantics.
• Comparing only total amount → Overcounts and omissions can cancel → Compare keys, counts, slices, and record differences too.
• Checking only whether the job succeeded → Success says the job ended, not that the data is correct → Add structural, business, and source-reconciliation assertions.
• Deleting duplicates directly in production → This destroys evidence and makes rollback unsafe → Preserve raw data and rebuild versioned output.
• Fixing the fact table without refreshing cache → Users still see old numbers and infer that repair failed → Invalidate relevant caches after the validated switch.
• Equating freshness with all quality → On-time data can still be duplicated or wrong → Define freshness, completeness, and correctness separately.
• Sending one generic alert for every defect → A message without dataset, slice, and owner cannot drive action → Include actual value, blast radius, and escalation path.
• Declaring recovery immediately after repair → The next incremental run may recreate the same defect → Observe a normal run and exercise the new guardrail.

Follow-Up Questions and Responses

Follow-up 1: Totals match, but order-level reconciliation still differs. Can you restore the dashboard?

Not from totals alone. An overcount on one order and an omission on another can cancel while customer, regional, or tax slices remain wrong. Continue comparing unique orders and event differences, and explain each class by state, currency, and refund relationship. Equal totals become one recovery signal only after allowed late arrivals and semantic exceptions are listed and critical consumers approve them.

Follow-up 2: Raw events have no stable event_id. How would you deduplicate them?

First request an upstream ledger, transaction ID, or replayable snapshot. A fingerprint built from time, amount, and user can wrongly merge two valid payments. If a composite key is the only option, specify fields, time tolerance, and conflict rules, measure false merges and missed merges in the shadow table, and keep an exception queue for review. If uniqueness cannot be proved, disclose the remaining uncertainty instead of calling the heuristic output an exact ledger.

Follow-up 3: The backfill needs six hours, but finance closes in two. What do you do?

Reduce scope by business impact. If the 8% gap is concentrated in one currency or the two hours after release, prioritize that partition and give finance the validated pre-release data, affected scope, and pending adjustment. If the backfill still misses close, finance should use the processor or another approved temporary reconciliation and post an adjustment later. Do not skip validation and replace the whole table with unverified output to meet the clock.

Follow-up 4: A one-to-many dimension join caused the error. How do you prevent recurrence?

Assert uniqueness of the dimension business key and non-overlapping validity intervals. Compare fact-key counts and row counts before and after the join, and impose a hard release gate on unexpected join multiplication. If the dimension must retain historical versions, the join predicate needs event time within the valid interval; joining every version by business key is invalid. Test boundary timestamps and overlapping-version fixtures.

Follow-up 5: Late refunds rewrite history every day. When is the dashboard correct?

Define two promises: fast visibility and final correctness. Show a timestamped preliminary net amount for the current day and define a refund-correction window. Late refunds inside that window trigger idempotent corrections; after the window, publish a near-final version. Refunds outside it enter a separate adjustment and audit process. The dashboard, data contract, and finance policy must use the same maturity state so a revisable number is not mistaken for a closed one.