Anomaly explanation by hand vs with an AI agent: what changes and what stays human

Explaining an anomaly by hand means one person quantifies the deviation, slices the metric by every plausible dimension, checks upstream changes and decides whether it is signal or noise, usually across four or five tools. An AI agent runs the same sequence through MCP tools and returns a plain-language explanation, so the engineer reviews a finished analysis instead of building one. The Model Context Protocol is what lets an agent in Claude Code or Cursor call those tools directly. The decision about what to do with the explanation stays with a human.

Key Takeaways

• •Detection is solved by most monitoring setups. The slow part is explanation: quantify the anomaly, attribute it to drivers, and judge whether it is real.
• •Done by hand, explanation is a sequence of queries and tool hops, and the engineer is the integration layer between them.
• •The dw-anomaly-explanation skill in our Incidents Agent does three things: it quantifies the anomaly, attributes drivers, and judges signal vs noise, then reports in plain language.
• •Fixes, backfills, stakeholder messages and policy changes still need a human approval. Explanation is the part worth delegating first.
• •Evaluate any agent for this job on replayed past anomalies where you already know the answer, not on a demo dataset.

Anomaly explanation by hand vs with an AI agent: the job and who does it today

The job starts after an alert fires. A metric moved, a row count dropped or a freshness score slipped, and someone has to answer the question the business actually asks: why? Typical prompts we hear from data teams are "Why did row counts drop 40% overnight?" and "Why did data quality score drop below SLA?"

Today that someone is usually an analytics engineer or the on-call data engineer. They are interrupted from other work, and the result is a Slack thread, not a durable artifact. Our own estimate, from pilot conversations, is that a single incident costs hours of human time between "something is wrong" and "it's explained and fixed". Treat that as our observation rather than an industry benchmark.

The manual workflow, step by step

The details differ by stack, but the shape is consistent. Each step has a handoff, and each handoff is a place where context gets lost.

• •Quantify. Pull the metric for the anomalous window and a comparable baseline, such as the same weekday over the last several weeks. Decide how large the deviation actually is in absolute and relative terms.
• •Slice. Break the metric down by region, product, channel, customer segment, source system and any other dimension that might concentrate the change. This is repeated ad hoc SQL, or clicking through a BI tool.
• •Check upstream. Look at what changed before the anomaly: a deploy, a schema change, a late or partial load, a failed retry, a new filter in a transformation. This means orchestrator logs, git history and the catalog.
• •Check the source. Ask the owning team whether the business really changed: a campaign ended, a region was switched off, a pricing change shipped.
• •Judge signal vs noise. Compare against normal variation, seasonality and known events. Decide whether this is worth anyone's time.
• •Write it up. Summarize the finding in language a stakeholder can read, and link the evidence.

The method is sound and has not changed in years. The cost is that steps 2 through 4 involve different systems, and nothing connects them. The engineer carries the context in their head, and when they are interrupted, they lose it.

Where the manual approach breaks

• •Slicing is open-ended. There is no natural stopping point, so people stop when they get tired or when a plausible story appears. Plausible is not the same as correct.
• •Confirmation bias creeps in. The first dimension that shows a concentrated drop often becomes the explanation, even when a second driver is hiding underneath.
• •Signal-vs-noise judgments are inconsistent. Two engineers looking at the same chart can reach different conclusions, and neither writes down the baseline they used.
• •The write-up is skipped under pressure. The next person to see a similar anomaly starts from zero.
• •The engineer is the integration layer between the warehouse, the orchestrator and the catalog. That is the point we make in our wider argument for a coordinated agent swarm over point tools.

What an agent does instead

Our Incidents Agent ships a skill called dw-anomaly-explanation. Its job, in the words of the skill description, is to explain a detected anomaly in plain language: quantify it, attribute drivers, and judge signal vs noise. We keep the claims to that scope on purpose.

• •Quantify: the agent measures the size of the deviation against a baseline and states it in plain terms, so the first sentence of the answer is a number, not a hunch.
• •Attribute drivers: the agent breaks the movement down across dimensions and reports which ones account for it, rather than stopping at the first plausible cut.
• •Judge signal vs noise: the agent says whether the movement looks like a real change or ordinary variation, and shows the reasoning.
• •Explain in plain language: the output is a readable explanation that a stakeholder can follow, not a query result to interpret.

Because every Data Workers agent is an MCP server, you invoke this from the terminal you already work in. In Claude Code or Cursor you ask the question as you would ask a colleague, and the agent calls its tools. The OSS edition (dw-claw) is Apache 2.0, runs locally and is read-only by default, which matters for this job: explanation needs to read data, not change it.

The agent also chains with others. Our Quality Agent monitors and raises the anomaly, and the Incidents Agent explains it. Questions that cross into schema, such as "What changed in the orders schema since last deploy?", draw on the same shared context instead of a second tool and a second login.

Manual vs agent: side by side

What still needs a human approval

An explanation is a read operation. Everything that follows from it is a decision, and we think most of those should stay with a person, at least until the team has watched the agent be right for a while.

• •Acting on the finding: backfilling data, re-running a pipeline, rolling back a change or dropping a record.
• •Declaring something a false alarm: suppressing or re-tuning an alert so it stays quiet is a policy change.
• •Telling the business: a stakeholder message about a number that moved should have a named owner.
• •Anything that touches PII or access: governance changes are not an anomaly-explanation concern.

In Data Workers this is the autonomy dial, set per domain. Pro adds writes, approval workflows and audit trails, so a fix can be proposed by the agent and approved by a person. Start with explanation at the lowest setting and raise it where the track record justifies it.

How to evaluate an agent for this job

The test is whether the agent reaches the right explanation on anomalies where you already know the answer. We suggest running it as follows.

• •Replay history. Pick 10 to 20 past anomalies with a known root cause from your incident log. Include some that were noise.
• •Hide the answer. Give the agent only what the on-call engineer had at the time.
• •Score three things separately: was the size right, were the drivers right, and was the signal-vs-noise call right.
• •Check the false-confidence cases. An agent that confidently explains a noise movement is worse than one that says "this looks like ordinary variation".
• •Read the evidence trail. Every claim should point to a query or table you can re-run. If you cannot verify it, you cannot trust it.
• •Check where it runs. Data should stay in your infrastructure, and the access should be read-only until you decide otherwise.

For dbt-based teams, the dbt documentation is a useful reference for what your existing tests already cover, so you know which anomalies the agent will see that your tests would not.

Frequently Asked Questions

What is anomaly explanation? It is the work between an alert and a decision: measuring how large the deviation is, finding which dimensions or upstream changes account for it, and judging whether it is a real change or normal variation. Detection tells you something moved; explanation tells you why.

Can an AI agent explain an anomaly without a human? It can produce the explanation end to end, and in a read-only setup that is safe. We still recommend a human approves any action that changes data, alerts or policy, which is why the autonomy setting is per domain rather than global.

Does the agent replace our monitoring? No. Monitoring detects and raises the anomaly, whether that is rule-based tests or statistical checks. The agent picks up from the alert and explains it. Our Quality Agent covers monitoring if you want both in one place.

Where do I run it? Anywhere an MCP client runs: Claude Code, Cursor, Windsurf, VS Code, GitHub Copilot or any compatible client. See Anthropic's Claude Code docs for how MCP servers are configured in Claude Code.

How should we start? Run the OSS edition locally in read-only mode on a handful of past incidents, score it using the checklist above, and move to Pro only if the explanations hold up and you want writes and approval workflows.