Data & Decisions
Use Data Quality Checks That Match the Failure You Need to Catch
Match each data check to a specific failure and downstream decision. Inspect a complete synthetic rule matrix and thirteen actual Python cases, including an empty output, blocked evaluations and an accuracy defect that still passes the specified rules.
Choose a data quality check by naming the failure it must catch and the decision its result will control. A non-null check cannot catch a duplicated key, a missing whole record or a fresh-looking but incorrect value. Specify the input scope, expected result, threshold, owner and failure response, then try a small known defect against the actual rule before trusting its green status.
This LandOffer.ai guide creates a complete check matrix and actually runs thirteen synthetic cases through an original Python checker. The catalogue, owners, dates and source manifest are fictional. No dbt project, Google Cloud scan, production pipeline, quarantine or notification system was run. The exercise verifies the authored checks against these inputs, including a deliberate blind spot; it does not certify a real dataset.

Start from a failure and its consequence
“Test every column for NULL” is a rule inventory. “Prevent a duplicated stock code from inflating the planning count” connects a failure to its downstream effect. The latter makes it possible to select a key, create a duplicate example and decide what must happen when the check fails.
The UK Government Data Quality Framework, published December 3, 2020, distinguishes completeness, uniqueness, consistency, timeliness, validity and accuracy. It also explains that complete data can still be inaccurate and that priorities depend on use. Treat these dimensions as prompts for reasoning, not a requirement to produce a score in every category.
Ask what an incorrect output would make someone do. A descriptive label missing from an internal planning page may permit a documented warning. A duplicated identifier that inflates a quantity may require holding the candidate batch. Those responses belong to the intended use; choose the response from that consumer's risk rather than importing a universal 99% threshold or blocking every optional omission.
Also identify the authority behind the expectation. An expected ID list, permitted location reference and source control total need versions and matching cutoffs. A total calculated from the same transformed rows can repeat the same defect. Record which trusted source boundary supplied the expectation and whether it refers to the same snapshot; otherwise reconciliation may compare two descriptions of the same bad output.
Worked asset: a filled pipeline check matrix
The fictional Larch print studio imports a read-only paper-stock planning catalogue. Its grain is one row per stock code, with one assigned location per code in this feed. A stipulated source manifest F1 expects codes C1–C4, permits locations N and S and reports 42 available packs. No real stock or purchasing decision is involved.
The baseline rows are C1/N/12, C2/N/0, C3/S/8 and C4/S/22. C2 has no optional description. Zero packs is valid and must survive the checks; the missing optional description produces a warning rather than a failed batch.
The fixed assessment time is November 10, 2026, at 12:00 UTC. Manifest F1 says the snapshot was created at 11:00 UTC. These future times are synthetic inputs. The fictional consumer needs a snapshot at most two hours old and tolerates up to five minutes of future clock skew. Those boundaries are planning assumptions, not observed service performance or universal standards.
| Rule | Failure to catch | Exact expectation and owner | Candidate-batch response |
|---|---|---|---|
| Q1 required fields | A row lacks its code, location or quantity | No null or blank code/location and no null quantity; engineer Romilly | Hold; retain failed row and field names |
| Q2 uniqueness | One stock code appears more than once | Each nonempty stock code appears once; Romilly | Hold; report duplicate keys and counts |
| Q3 numeric validity | A quantity has the wrong type or is negative | Integer, excluding booleans, and at least zero; analyst Dev | Hold invalid values; mark evaluation blocked if quantity is missing |
| Q4 reference validity | A location has no approved meaning | Every location is in F1's N/S reference; source owner Emrys | Hold unknown codes; do not map one by guesswork |
| Q5 record coverage | Expected codes disappear or unexpected codes arrive | Actual code set equals F1's C1–C4; Dev | Hold; list missing and unexpected codes |
| Q6 freshness | Snapshot is too old or implausibly ahead | Age between −5 and +120 minutes, inclusive, using manifest time; operator Una | Hold outside the range or on invalid timestamp |
| Q7 reconciliation | Valid row quantities do not match source total | Sum equals F1's 42 packs; Dev | Hold mismatch; block calculation if quantities are invalid or missing |
| W1 optional description | A display aid is unavailable | Missing description is recorded; product owner Celine | Warn without blocking this internal planning use |
This is an explicit rule set, not a weighted quality score. Any failed or blocked hard rule holds the candidate. W1 does not cancel a hard failure. Passing these rules means only that their specified expectations were met.
Separate row checks from whole-dataset expectations
dbt's data-test documentation describes assertions that return failing records, including generic non-null, unique, accepted-value and relationship tests, plus custom business logic. That pattern is useful: retain the actual violating records rather than a boolean without context. Our checker uses ordinary Python and does not claim to execute dbt SQL or validate a dbt configuration.
A rule that checks each row can find zero violations on an empty dataset. It has no rows to examine. In our empty-output case, required-field, uniqueness, numeric and location rules all report no row violations. Q5 nevertheless detects that C1–C4 are absent, and Q7 detects zero observed packs against the expected 42. The overall result is Hold.
Likewise, a code-set comparison does not enforce uniqueness. Adding a second C1 with zero packs leaves the distinct code set and control total unchanged. Q2 is needed to detect that extra row. These failures require different checks even when a dashboard groups them all under “completeness.”
Google Cloud's current auto data quality overview distinguishes row-level thresholds from aggregate conditions and SQL assertions. Its page, last updated October 26, 2026, identifies the product as Knowledge Catalog, formerly Dataplex Universal Catalog. We use the conceptual distinction, not its console, a cloud account or an assertion that our checker reproduces its behavior.
Worked asset: actual defects and their observed results
The Python 3.12.14 fixture copies the baseline before each mutation, so one case does not contaminate the next. Every input and per-rule result is saved. “Blocked” means the check could not make its promised evaluation; it is not silently counted as Pass.
| Synthetic case | Actual failed rules | Actual blocked rules | Overall result |
|---|---|---|---|
| Baseline, including valid zero and optional missing description | None | None | Specified hard rules pass; W1 warning |
| Set C2 quantity to NULL | Q1 | Q3, Q7 | Hold |
| Append another C1 with zero packs | Q2 | None | Hold |
| Set C2 quantity to −1 | Q3 | Q7 | Hold |
| Change C2 location to X | Q4 | None | Hold |
| Remove C3 | Q5, Q7 | None | Hold |
| Set snapshot age to 121 minutes | Q6 | None | Hold |
| Reduce C4 quantity from 22 to 21 | Q7 | None | Hold |
| Supply an empty row list | Q5, Q7 | None | Hold |
| Change C1 to 13 and C4 to 21 | None | None | Specified rules pass; accuracy blind spot |
Three additional boundary cases complete the thirteen: exactly 120 minutes old passes Q6; exactly five minutes in the future passes; six minutes in the future fails. The script verifies the complete expected failure and blocked-rule lists, not merely that some check happened to turn red.
The missing C3 case reduces the observed sum from 42 to 34 as well as changing the key set. One defect can trigger several rules. Conversely, the duplicated zero-quantity C1 proves that a passing total does not excuse a duplicated key. Preserve both results so the owner sees the missing record and its quantity consequence; do not assume each defect must correspond to exactly one check.
The final case is intentionally allowed through. Its quantities remain nonnegative integers, all expected keys remain present, location and time are valid, and 13 + 0 + 8 + 21 still equals 42. Without authoritative quantities for each code, this suite cannot identify the offsetting changes. It demonstrates a limitation, not a successfully detected accuracy failure. To catch it, the team would need trustworthy per-code quantities or another check capable of detecting the compensating changes. That added control has not been implemented or verified here, so the record must retain the blind spot.
Decide what to do with a failure before it occurs
For the missing-C3 fixture, Dev completes a fictional disposition: “Hold this candidate planning catalogue. The expected code C3 is absent and the subtotal is 34 rather than 42. Preserve F1, the supplied candidate and the result. Romilly investigates the transformation; Emrys confirms whether F1 and the input share the same cutoff. Emrys owns the reference-version decision, Romilly owns any implementation repair and Dev reviews new evidence. No replacement quantity or release is approved.”
This record does not diagnose a real transformation defect. A legitimate catalogue revision or mismatched manifest version could also explain a failed expectation. The right response is to reconcile the reference and input versions, not automatically loosen a threshold until the batch passes.
Celine separately accepts the baseline's optional-description warning for this internal planning purpose. That acceptance does not permit missing stock codes or quantities. If the same feed later drives a customer-facing catalogue that requires descriptions, the business use and W1 severity need a prospective reviewed change.
Define alert routing at the same level of detail. Una would route a hard failure to the named owner with rule ID, batch version, observed discrepancy and affected consumer. An optional-description warning would enter a reviewed work list rather than an urgent incident channel. These are completed fictional routing decisions; no alerts were sent or downstream job blocked by this article.

Test the blind spots and false alarms too
A passing freshness check establishes that the supplied timestamp falls within the rule. It does not prove that the content was regenerated correctly. A replayed old catalogue with a newly stamped time could satisfy Q6. A source-authenticity or content-reconciliation check would need additional evidence and is not implemented here.
Review thresholds with actual consumer needs and a relevant operating history when available. A rule set tuned only against one tiny baseline may fail on valid seasonal or operational changes. Keep a legitimate zero, a permitted null, an approved new-reference version and an exact-boundary example in the review packet when those cases apply. Do not invent historical false-alarm rates from this synthetic exercise.
When a rule changes, retain its prior version, the reason, approver and examples that should now pass or fail. Re-run the affected cases and review the consumer consequence. A changed threshold may be justified; an unexplained switch from Hold to Pass is not an investigation outcome.
Leave a specific acceptance boundary
Finish the packet with the source and candidate scope, rule versions, observed violations, blocked evaluations, warnings and named disposition. Keep “tests executed,” “specified rules passed” and “data accepted for this use” separate. Passing a suite is evidence about the suite's predicates, not a universal accuracy certificate.
Choose the next missing control from a real failure mode: expected-record coverage, a reference check, trusted timestamp evidence or per-key reconciliation. For more practical analysis records, continue with LandOffer.ai's workplace guides; keep the failure each rule can catch beside its green status.
Sources and Further Reading
- GOV.UK: The Government Data Quality Framework — December 3, 2020; fitness for purpose, quality dimensions and completeness versus accuracy.
- dbt: Add data tests to your DAG — assertion, failing-record and generic/custom test guidance; last updated September 16, 2026.
- Google Cloud: Auto data quality overview — current row-level, aggregate and SQL assertion distinctions. Current primary bodies checked October 27, 2026; no vendor installation or cloud account was tested.



