Why the data ecosystem needs a context layer for AI agents
What a context layer is, what breaks without one, and where we fit
The data ecosystem needs a context layer for AI agents because an agent that can run SQL still has no way to know what your tables mean, who owns them, what is governed or what a change will break. Agents connected through the Model Context Protocol get tools; they do not automatically get understanding. A context layer is the shared, queryable record of that understanding: schemas, lineage, ownership, policies, quality history and past incidents.
Key Takeaways
- •A context layer is a queryable record of what your data means, who owns it, how it flows and what policies apply to it. It is not a new warehouse and not a new dashboard.
- •Tool access (MCP) and context are separate problems. MCP standardizes how an agent calls a tool; it does not tell the agent which table is the source of truth.
- •Without shared context, every agent session re-derives the same facts from raw metadata, and the engineer becomes the integration layer between tools that do not talk to each other.
- •A context layer pays off most where agents act on shared infrastructure: schema changes, incident response, access reviews and cost questions. It pays off least on small, single-person projects.
- •Data Workers is one implementation of this idea, built as MCP-native agents. It is not the only way to build a context layer, and it is not worth adopting if your team has no agents touching production data.
Why the data ecosystem needs a context layer for AI agents
Consider a question our users actually ask: "Is the customer_id join safe after the migration?" A human data engineer answers it from memory and a few queries. They know that two tables share a key by convention, that one of them was backfilled in March, and that a finance dashboard depends on the old behavior. None of that lives in the warehouse schema.
An AI coding agent in Claude Code or Cursor starts from the schema and whatever it can read in the repository. It can inspect column types and run a count. It cannot tell you that the join silently drops late-arriving rows, because that fact lives in an incident ticket, a Slack thread and one engineer's head. Give the agent a better model and it still lacks the fact. The gap is missing context, not missing capability.
This is why we think the data stack needs a dedicated layer for it. Warehouses store data. Orchestrators run jobs. Catalogs describe assets. Each holds a slice of the truth, in its own format, with its own access model. An agent asked to reason across all of them has to stitch the slices together on every request, and it will stitch them differently each time.
What a context layer actually is
We use a narrow definition. A context layer is a service that agents query to answer four kinds of questions, with answers that are current, attributable and permission-aware.
- •Meaning: what does this column or table represent, and which one is the source of truth?
- •Lineage and blast radius: what depends on this, upstream and downstream?
- •Governance: who may see this, which columns hold PII, what retention applies?
- •History: what changed recently, what broke before, and how was it fixed?
It sits beside the data, not in front of it. Agents still read and write through your existing systems. The layer tells them what they are looking at before they act. This is the same idea people call context engineering: deciding what information reaches the model, in what form, at the moment it is needed. In a data team, most of that information already exists. It is just scattered.
What breaks without one
We see the same failures repeat when teams hand agents access to a warehouse and a repository and nothing else. These are our observations from working with data teams, not measured rates.
| Situation | Without a shared context layer | With one |
|---|---|---|
| Schema change in the orders table | The agent edits the model and passes local tests; a downstream consumer breaks later | The agent checks lineage first and reports the blast radius before editing |
| Row counts drop overnight | The agent queries the table repeatedly and guesses at causes | The agent reads recent schema, pipeline and quality events together and starts from a ranked hypothesis |
| Access question on PII columns | The agent lists grants from one system and misses the others | The agent answers from a unified view of classification and grants |
| Two agents working the same incident | Each re-derives the same facts and may disagree | Both read the same record and can see each other's findings |
The pattern is that detection and local action are easy and cross-system reasoning is hard. Our opinion is that this is the real bottleneck for agentic data work today, and that more capable models do not remove it, because the missing facts are not in the model's training data. They are specific to your company.
Tool access is not context
A common objection is that MCP already solves this. It solves a different problem. The MCP specification defines how a client discovers and calls tools, reads resources and uses prompts. Claude Code supports MCP servers, so an engineer can connect a warehouse, a repository and a ticketing system in minutes.
Connecting ten MCP servers gives an agent ten sets of tools and no shared model of how they relate. Whether a Snowflake table, an orchestrator task and a catalog entry describe the same asset is exactly what a context layer records. MCP is the transport; the context layer is what travels over it.
Where Data Workers fits, and where it does not
Data Workers is an open-source platform of fifteen MCP-native agents. The shared context is what lets them cooperate. Our Catalog Agent maintains the unified view of assets and meaning, and other agents read from it when they handle schema changes, incidents, quality and governance. Because every agent is an MCP server, an engineer reaches all of it from Claude Code, Cursor or any MCP-compatible client without opening another dashboard.
We covered how this plays out during an incident in our other resources: the Incidents Agent diagnoses root cause, the Schema Agent maps blast radius and the Pipeline Agent proposes the fix, all from one shared record rather than three separate investigations. In our early pilot testing, 60% of incidents auto-resolved before a human was notified. That is our own pilot number, from a small sample, not an industry benchmark.
The open-source edition runs locally and is read-only by default, so you can test the context layer on real metadata without granting write access. Writes, approval workflows and audit trails are part of the paid tiers. Our security model applies at the framework level, so access control, PII redaction and audit logging cover every agent and every tool call, and data stays in your infrastructure.
Here is where we are not the right answer:
- •Your team is two people and the warehouse has a few dozen tables. A well-kept README and dbt docs will serve you better.
- •No agents touch your data platform yet. A context layer has no reader, so start with the agent workflow first.
- •You need a business-intelligence semantic layer for metric definitions in dashboards. That is related but narrower, and we do not replace it.
- •You want a managed catalog with human curation workflows as the primary product. We are agent-first, and that is a different bet.
How to decide whether you need one
We suggest three questions. First, do agents or engineers using agents make changes that cross system boundaries, such as a schema edit that affects pipelines, quality checks and permissions? Second, do you see the same facts re-explained to each new agent session? Third, would an agent's wrong answer cost more than a day of cleanup? If you answer yes to two of three, a context layer is worth a trial.
Start small. Pick one question your team asks weekly, such as "What's the blast radius if I drop this column?", and test whether an agent answers it correctly with and without shared context. That experiment tells you more than any architecture diagram. Our open-source repository lets you run it read-only against your own metadata.
Frequently Asked Questions
What is a context layer for AI agents? It is a queryable record of what your data means, where it came from, who owns it and which policies apply, available to agents at the moment they act. It combines schema, lineage, governance, quality history and incident history into one view.
Is a context layer the same as a data catalog? Not quite. A catalog is a system of record that humans browse and curate. A context layer is designed to be read by agents, stays current from events rather than manual upkeep and spans more than assets, including incidents and policies. A catalog can be one input to it.
Doesn't MCP already give agents the context they need? No. MCP standardizes how agents call tools and read resources. It does not decide which information is correct, current or permitted. You still need something that holds and maintains that information.
Do we need Data Workers to build a context layer? No. You can assemble one yourself from your catalog, lineage and incident records behind an MCP server. We built Data Workers so teams do not have to build and maintain that integration, and so agents can share the same record.
Is it safe to let agents read production metadata? In our design, the open-source edition is read-only by default, and write actions in the paid tiers go through approval workflows with tamper-evident audit trails. Whatever you use, start read-only and widen access one domain at a time.