A customer says, 'Our dashboard is wrong,' but can't tell you which number. What do you do?
Answer it out loud first, then compare. That's what makes it stick.
I treat 'wrong' as a symptom to localise. I ask for one concrete example: which screen, which number, what they expected, and where that expectation came from. Then I trace that single number back through the pipeline: the metric definition, the query, the source system. Often it's a definition mismatch rather than a bug. I'd write down my findings, confirm with them, and only then propose a fix or a data contract.
- Get one concrete wrong number
- Ask what they expected and why
- Trace it: definition, query, source, refresh time
- Classify: bug, definition mismatch, stale data, expectation
- Confirm and agree a metric definition
A guide to the round lists this as a deliberately loose prompt: the customer cannot say which number is wrong. The test is whether you localise before you fix.
Step 1: Make it concrete. Ask for one example, ideally with a screenshot: 'Open the screen. Which number looked off? What did you expect, and where did the expected number come from (another report, a spreadsheet, memory)?' If they cannot say, ask when they last trusted it and what changed since.
Step 2: Check the cheap causes first.
- Freshness: is the data a day stale versus the source? Ask what breaks if an answer is a day old instead of a minute old.
- Filters and time zones: date ranges, fiscal calendars, currency, excluded segments.
- Duplicates or joins: a join fan-out can double a total.
Step 3: Trace one number end to end. Dashboard tile, then metric definition, then SQL or transformation, then source table, then the upstream system. At each hop, compare the value to the previous hop. The step where it changes is your suspect.
Step 4: Classify the finding.
- Real bug (bad join, failed job, schema change).
- Definition mismatch (finance counts 'active customer' as paid in the last 30 days, sales as any login). Most 'wrong dashboard' cases end here.
- Stale or missing data (late source, backfill needed).
- Expectation error (the other number was the one that was wrong).
Step 5: Fix the process, not just the number. Write a one-line definition per key metric with an owner, add a freshness indicator on the dashboard, and add 2-3 reconciliation checks (dashboard total versus source total) that alert when they diverge.
Tone: do not argue. 'Let's find one number we can both check' keeps the customer on your side.
What interviewers watch: whether you ask for evidence, whether you separate symptom from root cause, and whether you leave behind a check so it does not recur.
- Two teams each insist their number is correct. How do you resolve it?
- How would you catch this next time before the customer does?
- The customer is angry and wants an answer today. What do you say?
- What the forward deployed engineer interview really tests (prep, unknown) Lists the dashboard-is-wrong prompt as a typical loosely phrased decomposition question; round is about 45 minutes, usually no code.
- Decomposition interview guide (prep, unknown) Excellent answers separate root causes from symptoms and name trade-offs; example of diagnosing why an outcome missed its target by 40 percent.