How Do You Detect Breaking Schema Changes in dbt CI Before They Hit Downstream Consumers?
Combine model contracts, state selection and column-level lineage so a pull request fails before a dashboard does
To detect breaking schema changes in dbt CI, enforce dbt model contracts on every shared model, then add a column-level lineage check that walks downstream from each changed column. State selection alone is not enough: a September 2026 guide on lineage impact analysis in CI notes that comparing against a prior manifest "does not by itself prove which individual columns break." Contracts catch the shape change on the model you edited. Lineage tells you who consumes the column you changed.
Key Takeaways
- •dbt's
state:modified+selector decides which models to build and test, but it does not identify which columns a downstream consumer depends on. - •A dbt model contract fails the build when a contracted column is removed or its data type changes, and since dbt v1.6 removing or modifying a column constraint also counts as breaking.
- •A column-level lineage gate should traverse downstream from every changed column, use the production graph for removed or renamed columns, and label table-level-only hops as possible impact instead of inventing edges.
- •An empty lineage result from a stale catalog is not proof of safety, so the CI gate must record lineage coverage and fail closed when inputs are missing.
- •A data diff on the pull request shows value-level changes that contracts and lineage cannot see, such as a refactor that silently shifts a revenue calculation.
Why State Selection Alone Misses Breaking Columns
Most dbt CI setups start with dbt build --select state:modified+ --defer --state <prod-artifacts>. That compares your branch to a production manifest.json, selects every modified resource plus its descendants, and defers unchanged parents to production. It is the right way to keep CI fast. The same oneuptime guide recommends pairing it with --defer and --favor-state, and keeping the state directory separate from dbt's target directory so the comparison manifest is not overwritten.
The limit is granularity. state:modified+ works at the model level. If you rename customer_id to account_id in a staging model, the selector correctly rebuilds every descendant. It cannot tell you that exactly two of forty descendants reference that column, or that one of them feeds a finance model and a BI measure. Selection answers "what should we run?" The breaking-change question is "what will fail or return wrong numbers?"
That gap matters because a rebuild passing is a weak signal. A downstream model that selects * will rebuild fine after a column is dropped, and a BI tool outside dbt will not rebuild at all. The failure shows up later, in production, as a broken dashboard or a quiet change in a number. CI needs a check that reasons about columns, not just models.
Layer 1: Enforce Contracts on Shared Models
Model contracts are the cheapest layer and should come first. You set contract: enforced: true, then declare every column's name and data_type. Per the dbt documentation, dbt runs a preflight check that the query returns columns matching the contract, and the model fails to build if they do not. Column order does not matter.
Contracts also have a defined notion of breaking. The same documentation states that from dbt v1.6, removing or modifying a column constraint is a breaking change, and from dbt v1.9, removing a contracted model by deleting, renaming or disabling it is also breaking. Versioned models raise an error in that case, while unversioned models raise a warning. Removing a column or changing its data_type triggers a contract error when compared with the previous project state.
Contracts have scope limits. They apply to SQL models materialized as table, incremental or view, and not to Python models, ephemeral models, sources, seeds or snapshots. Put them on the models other teams and dashboards read, usually your marts and any model exposed across projects. Contracts protect the producer's own promise. They do not tell you who depends on what, which is the next layer.
Layer 2: Run a Schema Diff Against the Production Baseline
Contracts only cover models that opted in. A schema diff covers everything else. In CI, compare the column list and types of each modified model in the pull request build against the production baseline, and emit a structured record per change: added, removed, renamed, type-widened, type-narrowed, nullability changed. The record is the input to every later decision.
Classify each change before you evaluate impact. Adding a nullable column is usually safe. Removing a column, narrowing a type, or changing the semantics of a column while keeping its name are not. Semantic changes are the hard category, because the schema is identical and the meaning moved. Treat those as requiring an explicit owner approval, which the oneuptime guide also recommends for semantic changes.
Keep the diff deterministic and boring. It should read two artifacts, the production manifest and the branch manifest, and produce the same output every time. Anything that queries a live warehouse during the diff adds flakiness and credentials to a job that runs on every push.
Layer 3: Add Column-Level Lineage for Blast Radius
This is the layer that answers the target question. The oneuptime guide describes a gate that combines the schema diff with field-level lineage and traverses downstream from each changed field. For removed or renamed fields it uses the production baseline graph, so the candidate branch's lineage cannot erase the dependencies you are about to break.
Two details keep the gate honest. First, store a path record per hit, such as a staging model feeding a finance model and then a BI measure, so a reviewer sees the chain and not just a count. Second, when a hop only has table-level lineage, widen the result and label it POSSIBLE_IMPACT instead of fabricating column edges. The guide also notes that OpenLineage's Lineage Dataset Facet supersedes the older Column Lineage facet for the relationships it covers, so consumers should prefer the newer one while still reading older producers.
Our Schema Agent does this traversal from an MCP client: ask "what's the blast radius if I drop this column?" and it projects the downstream impact from the lineage the Catalog Agent federates. You can build the same gate with a script and your own lineage store. The point is that the answer comes from lineage, not from a person remembering who uses the column.
Layer 4: Add a Data Diff for Value-Level Changes
Contracts and lineage catch structural breaks. They do not catch a refactor that keeps every column name and type but changes values. A data diff compares a modified model's output on the CI branch with production and reports which rows and columns moved. The helpmetest write-up on Datafold in CI/CD gives an example of 10,000 rows in both environments with 47 rows differing (0.47%), all in the amount column.
The same article cites roughly 4 hours of investigation when a bad change reaches production and breaks downstream dashboards, and suggests guardrails such as blocking when more than 5% of rows change. Treat those thresholds as that article's examples, not universal defaults. Pick thresholds per model, since a 5% change in a dimension table is routine and a 0.01% shift in a revenue mean may not be.
Run the data diff only on modified models and their lineage-reachable descendants. Diffing the whole project on every pull request is expensive and produces noise that reviewers learn to ignore.
Layer 5: Turn Results Into a Pass, Review or Block Decision
The last layer is policy. The oneuptime guide proposes three rules: block breaking changes that have reachable production consumers, require review when lineage is incomplete, and require owner approval for semantic changes. It also says coverage must be part of the decision, meaning schema freshness, the share of jobs emitting lineage, and edge age. An empty result from a stale catalog is not proof of safety.
Make the evaluator fail closed. If the baseline artifact is missing, the lineage file is unreadable, or a migration exception has expired, the job should fail, not skip. Give exceptions an owner, a change ID and an expiry date, so a temporary waiver cannot quietly become permanent.
| Layer | What it catches | What it misses | Cost in CI |
|---|---|---|---|
| state:modified+ selection | Which models to build and test | Which columns break | Low |
| Model contracts | Removed or retyped contracted columns | Models without contracts, downstream consumers | Low |
| Schema diff | Every added, removed or retyped column | Who depends on the column | Low |
| Column-level lineage | Reachable downstream columns and BI consumers | Value changes with the same schema | Medium |
| Data diff | Row and value changes on the PR | Models that were not diffed | Medium to high |
A Practical Rollout Order
Do not try to ship all five layers in one sprint. Start with contracts on your ten most-consumed marts, since that costs a YAML change and gives an immediate failure signal. Add the schema diff next, because it needs only two manifests. Then wire lineage once you trust the coverage numbers, and add data diff for the revenue and customer models where silent drift is expensive.
Track coverage from day one. If only half your jobs emit lineage, the gate will say "no impact" for half of the changes it cannot see, which is worse than having no gate because it creates false confidence. Publish the coverage figure in the pull request comment next to the verdict.
Frequently Asked Questions
Does dbt detect breaking changes on its own?
Partly. With contracts enforced, dbt compares against the previous project state and raises an error when a contracted column is removed or its data_type changes, as the model contracts documentation describes. It does not know about consumers outside the project, such as BI tools, and it does not cover models without contracts.
Is column-level lineage required, or is model-level enough?
Model-level lineage tells you which models sit downstream. Column-level lineage tells you which of them actually read the changed column. Without it, you either block too often because every descendant looks affected, or you ignore the warning. When only table-level lineage exists for a hop, label the result as possible impact and require review.
What should the CI job do when lineage data is missing?
Fail closed. Missing or unreadable inputs and expired exceptions should produce a failing check or a required review, not a pass. Record lineage coverage in the output so reviewers can see how much of the graph the verdict is based on.
Where does a data diff fit if I already have contracts?
Contracts guard shape. A data diff guards values. A refactor can leave every column name and type untouched and still change numbers, which only a row-level comparison between the branch and production will surface.