Data Quality Monitoring — A Field Guide

Data quality monitoring exists because tests alone are not enough. A dbt test catches a known failure mode you already anticipated — a null where there should not be one, a duplicate key. Monitoring catches the failure modes you did not anticipate: a source system silently stops sending 20% of its usual volume, or a column's distribution shifts because someone changed a currency from cents to dollars upstream.

The five pillars people actually mean

When vendors talk about "data observability," they generally mean monitoring five things: freshness (is data arriving on schedule), volume (is the row count in the expected range), schema (did columns change type or disappear), distribution (do values look statistically normal compared to history), and lineage (which downstream tables are affected when something upstream breaks). Freshness and volume are the cheapest to implement and catch the most common real-world incidents — a source pipeline failing silently — so they are worth building first even without a dedicated tool.

Tests vs. monitoring — different jobs

dbt tests run at build time against rules you wrote: not_null, unique, a custom singular test checking that revenue never goes negative. They are deterministic and fast, but only catch what you thought to test for. Monitoring tools (Monte Carlo, Metaplane, Elementary, or a hand-rolled version using dbt's own `run_results.json`) watch metrics over time and alert on anomalies relative to historical baselines — useful precisely because you did not have to predict the failure in advance.

A freshness and volume check in dbt

dbt source freshness checks are the cheapest monitoring you can add — they require no new tooling, just a `loaded_at_field` declaration and a threshold.

sources:
  - name: raw_events
    database: raw
    tables:
      - name: orders
        loaded_at_field: _loaded_at
        freshness:
          warn_after: {count: 6, period: hour}
          error_after: {count: 24, period: hour}

# singular test: volume dropped more than 50% vs 7-day average
# tests/assert_order_volume_stable.sql
select current_count, avg_count
from (
    select count(*) as current_count
    from {{ ref('stg_orders') }}
    where order_date = current_date - 1
) c
cross join (
    select avg(daily_count) as avg_count
    from {{ ref('daily_order_counts') }}
    where order_date between current_date - 8 and current_date - 2
) a
where current_count < avg_count * 0.5

Alert fatigue is the real failure mode

The most common way data quality monitoring fails in practice is not a missed incident — it is too many low-signal alerts until people stop reading them. Anomaly detection on every column of every table produces constant noise on naturally variable metrics (weekend order volume is supposed to dip). Start monitoring on the handful of tables that actually feed executive dashboards or billing logic, tune thresholds against real history before expanding coverage, and route alerts to the team that owns the table, not a shared channel nobody is accountable for.

Wrapping up

Build freshness and volume checks first — they are nearly free with dbt and catch the majority of real incidents. Add anomaly detection only on the tables where a silent drift actually costs money, and be deliberate about alert routing, or the monitoring becomes noise nobody trusts within a month.