Schema changes that never break downstream
Upstream teams rename columns, widen types and drop fields without telling anyone, and the break shows up in a dashboard on Monday. Data Workers catches each change, traces every consumer it touches, and proposes a compatible migration with a rollback plan for a person to approve.
Before and after
| Today | With Data Workers |
|---|---|
| A renamed column found when a dashboard goes blank | Every schema change caught as it happens, with who or what made it |
| Impact traced by hand across dbt, BI and notebooks | Every affected model, dashboard and consumer listed before the change ships |
| Compatibility judged by whoever remembers the join | A compatibility check on every proposed change |
| Migration scripts written under pressure, no way back | A migration drafted with a rollback plan, approved by a named person |
| Owners learn about the damage afterwards | The affected owners are named before the change ships |
Most pipeline breaks start upstream, in a change nobody flagged. A schema evolution practice built on additive changes and compatibility checks works, but only if someone runs it on every change, every week. Drift detection on Snowflake tells you something moved; the harder job is knowing what it breaks and what to do about it.
How it works
1. Catch the change as it happens
The Schema Evolution agent watches your warehouse and sources with detect_schema_change and logs each renamed column, widened type or dropped field with list_schema_changes, including who or what made it. A change is on record before the first job fails.
2. Trace every consumer
The Data Context & Catalog agent runs blast_radius_analysis to list every downstream model, dashboard and consumer the change touches. This is the column-level impact analysis most teams skip because it takes days by hand.
3. Propose a compatible, reversible migration
check_compatibility tests the change against each consumer, and generate_migration writes the fix with a rollback plan. Nothing ships until a named approver signs off; agents start by watching and suggesting, and a schema change review becomes a decision rather than an investigation.
The impact metrics we move
- •Change impact mapped: from about 2 days to 15 to 30 min
- •Breaking change spotted: from about a day (when a dashboard breaks) to minutes
- •Migration with rollback plan drafted: from about a day to under 1 h
Where teams start
- •Warehouse tables behind executive dashboards. Watch the models finance and sales read first.
- •SaaS source feeds. Fivetran or API loads where vendors add and rename fields without notice.
- •Event and tracking schemas. Product instrumentation that changes every release.
- •dbt projects in CI. Run impact and compatibility checks on every pull request that touches a shared model.
- •Streaming topics. Kafka schemas registered, with their consumers listed.
What powers it
The Data-Agents Swarm does the work: the Schema Evolution agent detects and drafts, the Data Context & Catalog agent traces consumers through the Data Context Wizard, and the Quality Monitoring agent can re-check the affected tables afterwards. The Autonomous Data-Conductor coordinates the hand-offs, and Autopilot Standards keep a named approver on every breaking change. Spellbook Data Catalog keeps the record of what changed and who agreed. See how it shows up in reversible agent changes, end-to-end lineage and instrumentation QA per release.
The climb: from assisted to autonomous to agentic
Every transformation runs on the same ladder from our thesis. L0 to L2 run today: agents watch, trace and propose, and a person approves. L3 and L4 are where a domain can go on evidence, once its record of approved changes earns it.
The pilot
- •Scope. One domain: the schemas behind your most-read dashboards, plus the sources that feed them.
- •First week. Agents start by watching and suggesting. They log every schema change and show the impact and migration they would have proposed, with receipts.
- •By week two. Detection and impact mapping run on their own for the whole domain; every migration still goes to a named approver.
- •At day 30. We review the impact metrics against your baseline:
- •Change impact mapped: from about 2 days to 15 to 30 min
- •Breaking change spotted: from about a day (when a dashboard breaks) to minutes
- •Migration with rollback plan drafted: from about a day to under 1 h
- •Cost. Start with a pilot ($7,500 one-time, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.
FAQ
Does it apply migrations by itself? No. It drafts the migration and the rollback plan, and a named approver decides.
Which stacks does it cover? Warehouses such as Snowflake, BigQuery and Databricks, plus dbt, BI tools and 50+ connectors.
What if it misses a consumer? Impact comes from lineage, so consumers outside it can be missed. Gaps surface in the pilot, and the approver sees the list before anything ships.
Does it replace data contracts? No. A data contract states the agreement; this checks every change against it and everything else downstream.