Already on Databricks? What Data Workers Adds on Top of Genie, Unity Catalog and Agent Bricks

You standardized on Databricks. You probably still have a Snowflake share that finance depends on, dbt models, Airflow DAGs that predate Lakeflow, and Tableau or Power BI on top. The metric arguments and the 2 a.m. incidents happen where those systems meet, and Genie doesn't close those tickets.

On nearly every call with a Databricks customer, someone asks a version of the same question: "Databricks just launched Genie Ontology, Agent Bricks, managed MCP servers and an ops agent. Don't we already have all of this?"

Inside the workspace, largely yes. Genie One is the right tool for "what was revenue last week?" asked in Slack against a Unity Catalog table. But Genie is a coworker for the Databricks workspace. It's the wrong system of record for running an estate that also includes dbt, Airflow and another warehouse. Those teams don't have a question-answering problem. They have a close-the-ticket problem, and that's what Data Workers is built for.

Ask Genie about Databricks data. Ask Data Workers about the whole estate, and hand it everything that has to change outside the workspace and still be true.

Key takeaways

• •Data Workers turns your Databricks estate into an autonomous data platform. Cost optimization, migrations, incidents, governance and catalog upkeep run on their own, across every cloud you use, at whatever autonomy level you choose per domain.
• •Unity Catalog stays the lock on Databricks assets, and Genie stays the coworker in the workspace. Data Workers is the back office for the rest of the estate, and the review queue for anything that writes back into Databricks.
• •Databricks agents can read almost anywhere, but they can only fix things inside Databricks. Lakehouse Federation is read-only, Genie ZeroOps acts on Databricks assets only, and anomaly detection skips foreign tables.
• •Data Workers owns the loop across the estate: detect → diagnose → fix → review → verify → remember, running in Databricks, Snowflake, BigQuery, dbt, Airflow and your BI layer, with a receipt on every change.
• •Our catalog is a control plane, not an inventory. Spellbook Data Catalog is where agents' work is proposed, approved, rolled back and audited, and Data Context Wizard keeps one governed context graph across every platform.
• •The cost curves point in opposite directions. Genie One is free for human users until January 31, 2027, but service principals are billed from the first DBU, and agent work beyond that is metered in DBUs. Data Workers is a flat platform fee with unlimited seats and no usage meter.
• •MCP isn't the difference. Both sides speak it. The difference is where the operating model lives: inside one vendor's workspace, or neutral across all of them.

This guide compares what Databricks covers with what Data Workers adds. For the step-by-step path from a traditional data team to an autonomous data platform, read From Databricks to an autonomous data platform. For the leadership view, read You built on Databricks. Now make it agentic & autonomous.

Six things you get on top of Databricks

1. A data platform that runs itself, not just a smarter workspace. Genie makes the Databricks workspace agentic: people ask, and agents help. Data Workers goes further. The back office (governance, compliance, pipelines, catalog upkeep, cost and incidents) runs on its own, and your team moves up an autonomy ladder from observe-only to fully autonomous one domain at a time. Databricks' closest product, Genie ZeroOps, is in private preview and covers Databricks assets only. No platform vendor offers an autonomous data platform across a mixed estate today.

2. One control plane across every cloud, with no migration. Connect Unity Catalog, Snowflake, BigQuery, dbt, Airflow and your BI tools. Nothing moves. You get one context graph, one review inbox and one autonomy policy across all of them. Databricks federation lets you read other platforms. Data Workers lets you run them.

3. Cost optimization that never stops. The Cost Savings & Cleanup agent breaks your bill down by table, query, team and pipeline across Databricks, Snowflake and BigQuery. It finds the tables nobody has queried in months, idle and oversized compute, and duplicate pipelines, and it archives only after checking dependencies. Our design target is a 25–40% cut in warehouse spend. That's a target we're engineering toward, not a billed average. Databricks budgets can't see the Snowflake bill next door.

4. Migrations you approve instead of staffing. Hive and Glue into Unity Catalog, or Teradata, Oracle, Redshift and Snowflake workloads into the lakehouse. The Data Migration agent assesses the estate, translates SQL and procedural logic, re-translates anything that fails validation, and proves parity with row counts, checksums, statistical profiles and referential-integrity checks. It then cuts over incrementally while the legacy system stays live. Each wave is a single approval in Spellbook. Our target is 4–8 weeks for work that usually takes consultants 6–12 months.

5. Incidents closed, not triaged. The Autonomous Data-Conductor traces a broken number to its root cause in whichever system it lives in, has the right agent fix it there, and confirms the downstream number is right again. The goal is recovery in minutes instead of a war room, with a receipt on every change.

6. Governance and compliance by default. Access requests are provisioned through Unity Catalog grants with a receipt. PII classification stays current as tables change. Audit evidence is assembled from receipts instead of being hunted down every cycle.

Behind all six are 20+ specialist agents, one context graph across every platform, and the coding agent your team already uses (Claude Code, Codex or Cursor) as the way in. Spellbook is where you look: every asset, every change and every receipt, with one-click rollback.

One incident, five systems

Here's a scenario most Databricks shops will recognize (an illustration, not a customer case):

• •02:10. Fivetran picks up a renamed column in the Postgres billing source (cust_id becomes customer_id) and lands it in Snowflake.
• •02:40. The dbt model fct_revenue, which reads that table, still compiles but writes nulls for every new row.
• •04:00. A Lakeflow job copies the model's output into a Delta feature table the churn model trains on.
• •08:30. The CFO's Tableau revenue tile is down 6%. Someone asks Genie in Slack why.

Genie gives you the best explanation of the symptom. Data Workers closes the ticket. The rest of this guide is about that difference.

What Databricks covers, as of September 2026

It's worth being precise here, because a lot changed at Data + AI Summit 2026 and many of the names changed too. Databricks One became Genie One, Genie Spaces became Genie Agents, and AI Gateway became Unity Gateway.

That is a serious platform. If your entire data estate lives in one Databricks account, much of it is good enough, and we'll say so in the section on when Databricks alone is enough.

Here's the side-by-side. We scored eight outcomes a data leader actually buys. On three of them Databricks is on home ground: self-serve answers inside the workspace and agent tooling, where it leads today, and trusted context on Databricks data, where we're even because we ingest everything Unity Catalog and Genie know. The other five are the outcomes that cross the workspace edge.

The scores measure scope (what each side covers), not answer quality, and they're our directional judgments, not benchmarks. We've shown the reasoning so you can argue with any line.

Genie stops at the workspace edge

Every limit below comes from Databricks' own documentation. None of them is a bug. Each is a consequence of a sensible business decision: Databricks' agents exist to make Databricks the center of gravity.

1. The write path ends at the workspace edge

Databricks agents can read across your estate through federation and MCP. What they can govern and change is Databricks:

• •Query and catalog federation are read-only. Catalog federation also runs only on Databricks compute.
• •Genie ZeroOps fixes Databricks assets only: jobs, pipelines, tables and models. We found no documented capability for it, or any Databricks agent, to change a dbt model, an Airflow DAG, a Snowflake object or a BigQuery table as a governed product feature.
• •Anomaly detection runs only on Unity Catalog managed tables. Views and foreign tables aren't supported, and data classification doesn't list foreign tables either.

On a real enterprise estate, most incidents cross a boundary. A Fivetran schema change lands in Snowflake, breaks a dbt model, starves a Databricks feature pipeline and shows up as a wrong number in Tableau. An agent that can only act on one hop of that chain can describe the incident. It can't close it.

2. Context is learned from Databricks surfaces

Genie Ontology is Databricks' strongest launch this year. It learns from metric views, dashboards, SQL queries, Genie Agents, pipelines and connected apps, and it ranks snippets with a PageRank-style authority score. But it learns from what Databricks can see. We found no documented API for reading or writing the ontology directly, and no documented ingestion of dbt semantics, Snowflake semantic views or third-party catalogs into it. The practical result: your Snowflake team's definition of net_revenue and your Databricks team's definition can both be "authoritative" in their own systems, and nothing reconciles them.

3. Computed trust isn't governed trust

OntoRank weighs a definition's source, its author's authority, how often people use it, how close it sits to certified assets, and how fresh it is. That's a real signal. But popularity isn't correctness. The most-used definition of a metric can be the wrong one for the question being asked, and nothing in the launch materials describes a human approval step before a learned snippet starts steering answers at scale. We've written about this in more detail here.

4. Answers and proposals aren't owned outcomes

Genie answers questions. Genie Code helps an engineer who is sitting in a notebook. ZeroOps, once it ships, proposes fixes for Databricks assets. None of these is designed to own a class of work end to end: detecting it, fixing it wherever it lives, verifying that the fix held, and remembering what happened so the next occurrence is cheaper. That end-to-end loop is the product we build.

The overlap, honestly

Here is how we map the two side by side. "Use both" means Data Workers consumes the Databricks capability rather than competing with it.

What it costs as agents do more of the work

Genie looks free, and for a person asking questions it mostly is. Genie One and Genie Agents are free for human users until January 31, 2027, and every user gets 150 free DBUs a month. The meter shows up elsewhere:

• •Service principals are billed from the first DBU. Those are the identities automated agents run as.
• •Genie Code is billed once the free allowance runs out, under a serverless inference SKU.
• •Agent infrastructure runs on DBUs. Managed MCP servers bill as Databricks SQL or serverless compute, Supervisor Agent needs serverless compute, and anomaly detection and classification run on serverless DBUs.

As agents take on more of the operating work, that DBU line grows, including when the work is really about fixing a dbt model or a Snowflake table. Data Workers is priced the other way round: the Apache 2.0 core is free, Scale starts at $1,000 a month and Enterprise at $3,000 a month (billed annually), seats are unlimited, there's no usage meter, and there's no markup on model spend because you bring your own model key. Databricks compute stays on Databricks for Databricks work.

We're not claiming Data Workers costs less than Genie this quarter. During the promotion, a person asking Genie a question costs very little. The claim is that our cost doesn't grow with the amount of work agents do, and theirs does.

The fastest first win: Hive metastore to Unity Catalog

Every Databricks customer is being pushed off the Hive metastore, and it's the most concrete place to start. The Data Migration agent inventories your Hive and Glue tables, plans the move in reviewable batches, applies each batch through Unity Catalog, and verifies row counts, schemas and grants table by table. Every batch lands in the Spellbook inbox for approval and carries a rollback receipt. Data Workers speeds up your Unity Catalog rollout. It doesn't compete with it.

What each Data Workers product adds on a Databricks stack

Spellbook Data Catalog, the agentic catalog, vs. Unity Catalog

Unity Catalog is a governance and enforcement system. It's very good at deciding who can read which table on Databricks compute. A traditional catalog, UC included, is an inventory: it records what exists and who owns it.

Spellbook Data Catalog is built on a different premise. Once agents are doing the work, the catalog becomes the control plane where that work is proposed, reviewed, approved, rolled back and audited. We argue the full case in our whitepaper, From the Traditional Data Catalog to the Agentic Data Catalog. Concretely, on a Databricks estate:

• •Agents keep the metadata current. Descriptions, owners, lineage and quality status are written by agents and governed like any other change, instead of decaying the way manually curated catalogs do.
• •Asset pages cover the whole estate. A deep-wiki page for a Delta table shows its upstream Fivetran connector, the dbt model that feeds it, the Snowflake share that reads from it and the Tableau workbook that depends on it. UC's lineage graph can register external nodes (External Lineage went GA in June), but someone has to maintain them. In Spellbook, agents maintain them.
• •One inbox for agent work. Every proposed change, from any agent on any platform, lands in one queue: approve, steer, send back or roll back.
• •An authority guard enforced in code. No agent can promote its own work to authoritative or canonical, and a named human has to approve it. That is the difference between a catalog agents can safely write to and one they can quietly corrupt.
• •UC stays authoritative for Databricks. Spellbook doesn't mirror your grants into a second permission system. When an access change is approved, it's applied through Unity Catalog.

Spellbook is in preview today.

Data Context Wizard vs. Genie Ontology and metric views

Data Context Wizard is a living semantic and knowledge graph of your data layer, built for agents that can't ask a colleague what a column means. On a Databricks stack it differs in three ways:

• •It spans the estate. It covers Unity Catalog, Snowflake, BigQuery, dbt, Airflow, BI tools and existing catalogs, all in one graph, with 50+ connectors and no migration project. Your UC metric views go in as first-class sources.
• •Every fact carries a governed envelope: source, author, confidence, the time it was observed, and tenant. Facts are merged, not blindly appended, so a re-observed fact gains confidence instead of creating a duplicate.
• •Trust is scored and gated. We compute a multi-signal authority score, the same family of idea as OntoRank, and show a per-signal breakdown on every answer. The score orders the human promotion queue; it never promotes anything on its own. Databricks built the computed half of trust. We built both halves.

The graph also enforces PII scrubbing before storage, per-tenant isolation, tamper-evident audit, and encryption at rest with GDPR right-to-be-forgotten.

The Data-Agents Swarm vs. Genie Code and Agent Bricks

These are different kinds of product. Agent Bricks is a framework for building agents, and Genie Code is a copilot for an engineer. The Data-Agents Swarm is 20 specialized agents that already do the work: pipeline building, incident debugging, quality monitoring, schema evolution, change review, access and governance, security, identity, cost and cleanup, migration, observability, streaming, ingestion, MLOps, search and research, and more, across 50+ integrations.

For a Databricks team, a few matter most:

• •Schema Evolution agent. It catches an upstream contract change before it breaks a Delta table and the dbt models and dashboards downstream of it.
• •Data Migration agent. Hive metastore to Unity Catalog is the migration every Databricks customer is being pushed toward. The agent plans it, runs it in reviewable batches, and verifies parity.
• •Data Change Review agent. It reviews pipeline and model changes for blast radius before they merge, across repos rather than only inside the workspace.
• •Cost Savings & Cleanup agent. It finds unused tables, over-provisioned jobs and duplicate pipelines across every platform you pay for, not just the one it runs in.

The swarm is callable over MCP. If your team builds with Agent Bricks, a Supervisor Agent can call Data Workers agents as tools, so you don't have to choose.

Autonomous Data-Conductor vs. Genie ZeroOps and Lakeflow Jobs

Genie ZeroOps is a validation of our founding thesis: data operations should run on a detect-to-resolve loop, not on a pager. It's also confined to Databricks assets, and it's in private preview.

Autonomous Data-Conductor is the orchestrator that owns an outcome rather than a step. It runs detect → diagnose → fix → review → verify across the estate, directs the right agents in the swarm, scopes the blast radius before acting, and leaves a signed receipt on every change with one-click reversal. The last step matters most. A fix isn't closed until the Conductor has confirmed the downstream effect: the dashboard number is right again, the SLA is met again. What happened is then written back to the context graph, so the next occurrence resolves faster.

Autonomy guardrails and security

Databricks has made real progress on agent governance. Unity Gateway puts spend caps and guardrails on models and MCP servers, and Contextual Service Policies can require approval before an agent calls a tool. Those are guardrails at the tool-call level, inside Databricks.

Data Workers governs at the level of work across the estate, and it's designed so you can move up the autonomy scale gradually:

• •Autonomy is set per domain, not per product. Access provisioning can run at L3 while net-new pipeline builds stay at L2.
• •New deployments start observe-only. You extend autonomy one domain at a time as the receipts earn trust.
• •Every write is scoped before it runs, with blast radius computed across platforms, not just inside one workspace.
• •Every change leaves a signed receipt and can be reversed in one click.
• •No agent can approve or promote its own work. This is enforced in code, not left to a prompt.
• •Least privilege. Data Workers acts with the grants you give it. On Databricks those are Unity Catalog grants, so UC's enforcement still applies to everything we touch there.

"Databricks agents can use MCP and reach other tools. Doesn't that close the gap?"

Partly. We'd rather say so than pretend otherwise. Databricks agents can call external MCP servers, and external agents can call into Databricks. "Cross-platform because we speak MCP" is not a defensible differentiator for anyone anymore, including us.

The real question is where the center of gravity lives. In the Databricks model, external systems get registered into a Databricks-governed environment: memory lives in Lakebase, permissions flow through Unity Catalog, spend flows through Unity Gateway, and the agents act outward from the workspace. That's a coherent design if Databricks is your data estate.

Data Workers is built for the more common case, where no single platform is the center. Snowflake owns one workload, Databricks another, dbt the transformations, Airflow the scheduling, and a separate catalog and BI layer on top. We maintain one operating context across all of it, and we run the loop wherever the work is. An MCP connection gives an agent one more tool. It doesn't give you a coherent, continuously updated operating model of your whole estate. That model is what we build.

How it fits together

Getting started takes no migration. You connect Data Workers to Unity Catalog, your other warehouses, your dbt project and your orchestrator. The Context Wizard builds the graph, every agent starts in observe-only mode, and the first things you see are the cross-platform lineage and incident history your Databricks views couldn't show you. From there, you decide which domains earn more autonomy.

When Databricks alone is enough

We'll say this plainly, because you'll figure it out anyway:

• •Your whole estate is in one Databricks account. No Snowflake, no BigQuery, and dbt and orchestration running inside Lakeflow.
• •Your only need is Q&A on data that never leaves Databricks. Genie One and Genie Agents do that well.
• •You're happy to build your own ops agents with Agent Bricks and wait for ZeroOps to go GA.

Data Workers earns its place when any of these is true: more than one platform sits in the critical path; dbt, Airflow or BI changes cause your incidents; you want the back office (governance, compliance, pipeline management, catalog upkeep) to run autonomously rather than be assisted; or you need graded, reversible autonomy with receipts your auditors will accept.

FAQ

Does Data Workers replace Unity Catalog? No. Unity Catalog stays the enforcement point for Databricks assets. Spellbook Data Catalog is the agentic control plane across your estate, and it applies approved changes through UC.

How is Data Context Wizard different from Genie Ontology? Genie Ontology learns from Databricks surfaces and ranks context by computed authority. Data Context Wizard spans every platform, carries provenance on every fact, and puts a human approval gate in front of anything that becomes authoritative.

Can Agent Bricks call Data Workers agents? Yes. The swarm is MCP-native, so a Supervisor Agent can use Data Workers agents as tools.

Is Data Workers open source? The core is Apache 2.0 and free to run. See pricing for what the enterprise platform adds.

Does Data Workers slow down our Unity Catalog rollout? No. The Migration agent speeds up the Hive and Glue move into Unity Catalog, and every change Data Workers makes on Databricks goes through UC grants.

What happens if an agent gets something wrong? Every write is scoped before it runs and goes to review at whatever autonomy level you've set for that domain. It leaves a signed receipt and can be reversed in one click. No agent can approve its own work.

What does Data Workers store? Metadata and scrubbed facts about your data (definitions, lineage, owners, incident history), not copies of your tables. Queries run through the credentials you grant, PII is scrubbed before anything is stored, tenants are isolated, and Enterprise can run in your own VPC or on-premise.

Sources for Databricks capabilities and statuses: Databricks documentation, release notes and blog posts current as of September 29, 2026, including Genie Ontology docs, Introducing Genie ZeroOps, query federation, anomaly detection, managed MCP, Unity Gateway and Agent Bricks. Product names and statuses change quickly; if we've got something wrong, tell us and we'll fix it.