Data & Decisions
Two Dashboards Disagree: Reconcile the Metric Definition Before Fixing the Chart
A synthetic order dataset explains why two reports show six and three, then reconciles grain, paid-status filtering, New York day boundaries, and snapshot timing with local Python and SQLite checks.
When two dashboards disagree, reconcile what each number counts before changing the chart. Compare the metric definition, data grain, identity key, timestamp, time zone, filters, aggregation, and data snapshot. Trace a small set of records through both calculations. Check whether the two reports answer the same question; neither a matching title nor a preferred number establishes that.
This LandOffer.ai guide includes a completed metric-definition comparison and an actual offline reconciliation of original synthetic order data. We ran local Python and SQLite checks; no Amplitude, Looker, warehouse, business account, or live dashboard was accessed or changed. Documentation was checked October 27, 2026. The example demonstrates how a difference can be explained, not that either product or a real organization's reporting is incorrect.

Freeze the questions and the snapshots
Capture both reports as they were when the disagreement was observed. Record their exact filters, date boundaries, saved time zone, metric expression, grouping, source model, last successful data load, query or cache time, and relevant permissions. Relative dates such as “yesterday” need an evaluation time. Capture those conditions with the screenshot so another analyst can reconstruct the comparison.
Write the business question in plain language. “Orders” could mean order-line rows, distinct order identifiers, paid orders, orders created that day, or orders paid that day. Each can be a useful measure, but they cannot share a definition merely because a tile uses the same short label. Confirm the intended decision with the metric owner rather than letting the current chart select the meaning.
Amplitude's Funnel Analysis FAQ provides a named example of legitimate analytical differences: funnel and event-segmentation charts apply different qualification and filter rules, so choosing the same events does not necessarily produce the same result. That illustrates why the analysis definition matters. This article's order-count example does not use an Amplitude funnel or reproduce its calculation engine.
Keep original evidence while building a comparable view. If one report intentionally answers a different question, rename or explain it instead of forcing the values to match. If the agreed definition is violated, document the defect at the transformation where it occurs. Locate the population difference before editing the visual; its format does not change which records were counted.
Check grain, time, and freshness separately
Inspect what one source row represents. Joining orders to line items can create several rows for one order without creating several orders. Count the appropriate identifier at the intended grain, and investigate unexpected duplication before using a distinct count as a blanket fix. Otherwise a convenient aggregation may conceal an incorrect join or conflicting records.
Identify the exact timestamp field that defines membership in the reporting period. Creation time, payment time, ingestion time, and later status changes answer different questions. Specify an inclusive start and exclusive end, including the time zone. Use the named zone and date-aware conversion rather than assuming a fixed offset always describes the local day.
Looker's time-zone documentation explains how query settings can convert time-based results and filters, and why different settings can change values for a period. Its caching documentation separately explains that queries can use cached results and that cache policies can be coordinated with data loading. These are documented Looker mechanisms, not a diagnosis of the fictional reports below. We have not inspected any account settings or cache behavior.
A newer query does not guarantee newer underlying data. Check ingestion, transformations, persisted models, and cached results as distinct stages. If a row has not arrived in the source snapshot, refreshing a visualization alone cannot count it. Conversely, a cache difference needs evidence; do not blame caching simply because the numbers disagree.
Completed asset: the two definitions do not match
Our synthetic dataset contains eight order-line rows representing seven distinct orders. O02 has two legitimate line items; it is not an accidentally duplicated source order. All statuses are fixed illustrative values. The dataset does not contain a status-change history and cannot reconstruct whether an order was paid at an earlier point in its lifecycle.
Imagine two report specifications, both titled “Orders — October 26.” Report A counts order-line rows created during the UTC date, includes every status, and uses a snapshot through October 27 at 06:00 UTC. Report B counts distinct paid-status orders created during the America/New_York date and uses a snapshot through 05:00 UTC. These reports are fictional specifications replayed locally, not screenshots or observations from a reporting product.
| Definition field | Report A | Report B |
|---|---|---|
| Counted unit | Rows at order-line grain | Distinct order_id |
| Qualification | All listed statuses | Status equals paid |
| Date field | Order creation timestamp | Order creation timestamp |
| Calendar window | October 26 in UTC | October 26 in America/New_York |
| Snapshot eligibility | Ingested by October 27, 06:00 UTC | Ingested by October 27, 05:00 UTC |
| Aggregation | Row count | Distinct order count |
| Local replay result | 6 | 3 |
The same tile title hides four differences: grain, status, day boundary, and snapshot cutoff. Neither value is comparable until those conditions are aligned. Report A's six is a row count for its specified window, not six distinct orders. Report B's three is a distinct count in an older snapshot, not necessarily the complete later count for that local date.
For this particular date, our Python zoneinfo conversion maps New York's October 26 day to the UTC interval starting October 26 at 04:00 and ending October 27 at 04:00, with the end excluded. That is a date-specific result, not a rule to subtract four hours on every date. The retained script records the zone key, runtime, input hash, boundaries, and results.
The full original record set makes the differences inspectable. Times below are UTC; ingestion is when the row becomes eligible for the synthetic snapshot, not when the order was created.
| Order | Created | Status | Line rows | Ingested |
|---|---|---|---|---|
| O01 | Oct 26, 00:30 | paid | 1 | Oct 26, 00:31 |
| O02 | Oct 26, 05:00 | paid | 2 | Oct 26, 05:01 |
| O03 | Oct 26, 12:00 | paid | 1 | Oct 26, 12:01 |
| O04 | Oct 26, 18:00 | pending | 1 | Oct 26, 18:01 |
| O05 | Oct 26, 23:30 | cancelled | 1 | Oct 26, 23:31 |
| O06 | Oct 27, 02:00 | paid | 1 | Oct 27, 02:01 |
| O07 | Oct 27, 03:00 | paid | 1 | Oct 27, 05:30 |
O01 falls on the previous New York local date even though its UTC creation date is October 26. O06 and O07 fall within New York's October 26 day but outside the UTC date. O07 is also absent from the older 05:00 snapshot because it is ingested at 05:30. Those are separate conditions, and the reconciliation preserves both.
Completed asset: follow the difference through each stage
We replayed the specifications over the CSV with Python 3.12.14. Starting from A, we changed one condition at a time and retained the selected order and line identifiers at every stage. The resulting bridge is:
| Replay step | Result | Explanation of the change |
|---|---|---|
| A: UTC, all statuses, line rows, 06:00 snapshot | 6 | O01–O05 produce six line rows because O02 contributes two |
| Switch to distinct orders, keeping the other conditions | 5 | O02 contributes once rather than twice |
| Restrict those UTC orders to paid status | 3 | O04 and O05 are excluded; O01, O02, and O03 remain |
| Use the New York day, keeping paid status and the 06:00 snapshot | 4 | O01 leaves; O06 and O07 enter; O02 and O03 remain |
| Use B's older 05:00 snapshot | 3 | O07 has not arrived; O02, O03, and O06 remain |
The arithmetic is 6 minus 1 minus 2 plus 1 minus 1, yielding 3. This is one chosen reconciliation path, not a claim that the factors have independent causal contributions. Changing the order of the comparison can change the size of intermediate differences because eligibility conditions interact. Keep the record membership with the number so the bridge can be checked.
Notice that two intermediate results both equal three but contain different orders. The UTC paid set is O01, O02, O03; B's set is O02, O03, O06. Equal totals alone would have hidden that disagreement. Compare identifiers and boundaries before declaring the problem resolved, especially when offsetting errors could cancel in an aggregate.
We also ran a separate in-memory SQLite distinct-order query over the original rows, using the computed New York UTC bounds, paid status, and the 06:00 cutoff. It returned O02, O03, O06, and O07, matching the Python result of four. The retained SQL, bound parameters, CSV hash, runtime versions, and selected identifiers support that narrow offline check. This does not validate a live warehouse model, permissions, ingestion guarantee, or reporting application.

Agree on the metric before changing either report
In the fictional business review, metric owner Priya accepts this definition: “Distinct orders created on the America/New_York calendar date whose status is paid in the stated reporting snapshot.” The timestamp is order creation, the key is order_id, the qualification is paid status, and the snapshot cutoff must be displayed. It is not a measure of payments received that day, revenue, or historical paid status at creation.
For the completed example, the chosen reporting snapshot is October 27 at 06:00 UTC. The result is four, with the identifier set confirmed by the two local implementations. That choice is explicit within the fiction; the actual local calculation does not decide what a business should measure. An organization needing status as of an earlier moment would need appropriate historical data and a different contract.
Priya's recorded disposition separates semantic decisions from implementation: “Use the agreed distinct paid-status creation-date definition for the shared order metric. Preserve the older snapshot as evidence. Request the report-model and label changes, and have the owners verify the resulting records and freshness before marking the dashboard correction complete.” No report change, refresh, or business publication occurred for this article.
If Report A's line-item volume remains useful, give it a different name and definition. If Report B remains available before all expected arrivals, show its cutoff and completeness limitation. Do not silently rewrite a previously distributed figure without a correction note explaining the changed definition or data availability. A matching later total does not erase the earlier context.
Verify the correction from source to presentation
For a real repair, check the authorized source snapshot, transformation grain, metric expression, filter application, timezone boundary, and rendered label. Verify expected included and excluded identifiers alongside the total so offsetting errors cannot silently pass the comparison. Test boundary records and relevant status cases. Confirm the report is using the intended model version and that the visible freshness information matches the data it actually queried.
A refresh can change data without changing the definition; a model correction can change the definition without adding a row. Record which happened. If the business adopts a revised metric, version its contract and explain whether past reports will be restated. Keep unresolved data quality, access, or historical-status questions with an owner rather than labeling every remaining difference a chart defect.
Use the comparison sheet and record-level bridge to produce an explanation another analyst can reproduce. LandOffer.ai publishes this data-work guide; it does not inspect or repair your dashboards. Leave users with the agreed definition, record-level reconciliation, and the actual implementation status, including limitations that could change the number.


