You're on Telmai: It Trips the Breaker on Bad Lake Data. Data Workers Fixes the Fault and Resets It
Telmai validates data at the source and in the lake, bins bad records and trips the circuit breaker. Data Workers diagnoses, fixes and verifies behind approvals.
Your data platform team runs on Telmai. It profiles every attribute of the data in your lake and lakehouse, on raw files and open table formats such as Apache Iceberg, without sampling; out-of-box monitors learn what normal looks like while record validation rules check each row. Data Binning splits a batch into valid and invalid paths on S3, so costly pipelines only run on healthy records, and a circuit breaker built on Telmai's scan and alert APIs stops the run when an incident is serious. Release 26.3.4 (September 16, 2026) added a hybrid deployment model: Telmai hosts the control plane and scans run on your own Spark. Telmai's seven Data Reliability Agents (orchestration, validation, incident diagnosis, lineage, data insight, help and routing) are introduced on its homepage with an early-access waitlist, and the Telmai MCP server brings incidents, monitors and lineage into Claude and Cline. Telmai is the circuit breaker on your lake. Data Workers is the electrician.
When the breaker trips, the bad batch is held and the business is protected, but the fault is still there. Someone has to find what changed upstream, fix the job, update the rule, rerun the pipeline and prove the numbers. Data Workers does that work behind approvals and leaves a receipt.
Key takeaways
- •Telmai keeps its job. Profiling, monitors, rules, Data Binning, the circuit breaker and the Telmai MCP server stay where they are.
- •A tripped breaker arrives with a cause and a fix. Data Workers reads the Telmai incident, traces the fault through DMS, Step Functions, Iceberg and Athena, and proposes the job change, rule update and rerun as one change set.
- •A named owner approves; Data Workers verifies. After approval in Spellbook, Data Workers queues the rerun through Step Functions and checks volume, quality and load lag against its baseline on every table the fix touched.
- •Autonomy per domain, from L0 manual to L4 autonomous, at the pace your team sets.
Telmai is the circuit breaker on your lake. Data Workers is the electrician.
A breaker cuts the circuit before the fault burns the house down. The electrician finds the fault, fixes the wiring, resets the breaker and tests every outlet on the circuit. In data terms, Telmai is the smoke alarm and the breaker; Data Workers is the crew.
Here is a Friday at a property and casualty insurer on AWS: Amazon Aurora PostgreSQL (policy administration), AWS DMS, S3, Telmai, an AWS Step Functions state machine, EMR Serverless Spark, Apache Iceberg tables in the AWS Glue Data Catalog, Amazon Athena, Amazon QuickSight and Jira (times ET). The state machine premium_nightly calls Telmai's scan and alert APIs as its circuit-breaker step before any merge. This is an illustrative example.
| Time | System | What happens |
|---|---|---|
| Thu 16:00 | Aurora PostgreSQL | A billing release adds installment_count to policy_premium and, for monthly-billed policies, starts writing the per-installment amount into premium_amount, which used to hold annual written premium |
| Fri 00:30 | AWS DMS | The CDC task lands 9,214 changed policy rows, new column included, as Parquet under the raw prefix in S3 |
| 01:00 | Step Functions | premium_nightly starts; its first state triggers a Telmai scan of the batch and polls the job |
| 01:18 | Telmai | The record validation rule premium_amount >= 300 fails on 8,906 rows, all monthly-billed; the drift monitor flags mean premium down 27%. Data Binning writes the 8,906 rows to the invalid path, the incident opens at HIGH, Slack posts and a Jira ticket opens from a ticket template |
| 01:19 | Step Functions | The circuit-breaker state reads the HIGH alert and stops the execution before the Spark merge; silver.policy_premium and the gold earned-premium table stay on Thursday |
| 01:25 | Data Workers | The team's assistant hands over the Telmai incident; Data Workers reads the halted execution from Step Functions. The incident's schema drift signal names the new installment_count column, first seen in the 00:30 DMS batch; the failing rows' premiums sit at about one twelfth of each policy's prior value. The context graph holds the definition: premium_amount is annual written premium. Blast radius: the Iceberg silver table, the gold earned_premium_daily table built by Athena, the QuickSight "Earned premium" dashboard and the 07:00 actuarial reserving extract |
| 01:40 | Data Workers | Proposes one change set: a diff to the Spark merge job that annualizes premium_amount when installment_count is above 1, an updated validation rule drafted for the Telmai admin that checks the annualized value, a rerun of premium_nightly for Friday's batch, and a rollback to the pre-merge Iceberg snapshot, run by the table owner, recorded as the undo. The approval request goes to the premium data owner in Slack |
| 05:50 | Spellbook | The premium data owner reviews the diff, the definition it follows and the blast radius, approves and merges the Spark change; the Telmai admin applies the updated rule |
| 06:00 | Step Functions | Data Workers queues the rerun; Telmai rescans the raw batch, every row lands on the valid path and the breaker stays closed; the Spark merge and the Athena gold build run |
| 06:35 | Telmai + Athena | Data Workers verifies: the owner's Athena row count matches the DMS batch, the premium distribution in the new Telmai scan is back inside its band, load lag on silver and gold is back inside its monitor_metrics baseline, and the next Telmai scan passes |
| 06:50 | QuickSight and Jira | The scheduled dataset refresh reads the corrected gold table; the 07:00 reserving extract runs on time; the owner closes the Jira ticket with the receipt link |
Telmai did exactly what it is built for: it caught the bad batch at the source, kept 8,906 rows out of the lake and stopped the pipeline before the gold table understated earned premium. What changed is everything after the trip: a cause traced two systems upstream, a fix that follows the column's business definition, one change set to approve, a rerun, verification on every table it touched and a recorded undo.
| Job | What Telmai does | What Data Workers does |
|---|---|---|
| Detection | Profiles every attribute without sampling; monitors and record validation rules catch drift, completeness and correctness issues on lake files and open tables | Tracks freshness and volume against monitor_metrics baselines and reads Telmai's drift signals, so an upstream schema change arrives with its cause and blast radius |
| Containment | Data Binning splits valid from invalid records; the circuit breaker stops the pipeline through the scan and alert APIs | Treats the binned rows and the halted run as evidence and joins them to the source change that caused them |
| Root cause | Root cause analysis (Advisory) proposes a likely cause with its reasoning; lineage shows the asset's parents and dependents | Joins the incident to DMS batches, Step Functions runs, schema changes, business definitions and incident history to find what changed and what else it touched |
| Routing | Slack notifications and Jira tickets, created when an incident opens or by hand | Routes the approval for the fix to a named owner in Slack or email; unanswered requests expire and escalate |
| The fix | Recommends rules; the MCP server can create and change monitors on request | Proposes the complete repair (job diff, rule update, rerun, undo) with blast radius as one change set |
| Running it | Your team edits the job and restarts the pipeline | The owner approves and merges; Data Workers queues the rerun through Step Functions |
| Verification | The next scan passes | Checks volume, quality and load lag against its baseline on every downstream table, plus the next Telmai scan |
| The record | The incident and its linked Jira ticket | A receipt: the cause, the diff, who approved it, what it touched, how it was verified and how to undo it |
Why doesn't Telmai just do this itself?
Because Telmai drew its line on purpose and wrote it down. Its AI governance page classifies every AI capability as Advisory, Applied and reviewable, or Deterministic detection, and states: "No classification permits writing to, modifying, or deleting data in your source systems. That is prohibited at every level." Root cause analysis and rule recommendations are advisory: "Nothing changes unless you act on it." Monitor management through the assistant applies only to Telmai's own configuration, and Telmai's stated direction is to move it to a propose-and-confirm step. Even the circuit breaker is a pattern your orchestrator runs on Telmai's APIs.
That is the right design for a data quality platform. Telmai can be pointed at every bucket and Iceberg table in a regulated insurer because nobody worries it will change a row. Editing a Spark job, rerunning a pipeline and proving a reserving extract is a different product with a different liability: blast radius, the owner's approval, an undo before it runs, steps in order, proof and a record, in systems Telmai validates but does not run. Data Workers is built for exactly that job.
Every tool owns a slice. Data Workers covers the whole lifecycle
Telmai owns data quality at the source and in the lake, and it is one of the strongest tools for validating raw files and open table formats before they reach a warehouse. Each point tool adds another console, another contract and another handoff. Data Workers covers the whole lifecycle with one context, one approval flow and one audit trail.
| Stage | Data Workers | Telmai | Why we scored it this way |
|---|---|---|---|
| Catalog & Context | 9 | 3 | Telmai shows asset lineage and integrates with Atlan, Alation, Actian and Unity Catalog metadata. Data Workers keeps one governed context graph of definitions, owners, lineage, quality and usage across every platform. |
| Analytics & Insights | 8 | 2 | The Data Insight agent (early access) and the MCP server answer questions about data health. Data Workers answers data questions from governed definitions with lineage behind every number. |
| Data Quality | 8 | 9 | Telmai's home stage: profiling, out-of-box and user-defined monitors, record validation rules, Data Binning and DQ scores, run on lake files and open table formats without sampling. Data Workers writes and runs checks and proposes the tests behind them. |
| Observability & Incidents | 8.5 | 7 | Incidents group alerts per asset, with root cause analysis, Slack and Jira tickets. Data Workers diagnoses across systems, fixes with approval and verifies the fix. |
| Pipelines & Ingestion | 8.5 | 3 | The circuit breaker stops a pipeline through Telmai's scan and alert APIs. Data Workers queues reruns through your orchestrator, with approvals, and catches upstream changes first. |
| Schema & Migration | 8 | 3 | Schema drift is one of the health signals Telmai watches. Data Workers takes schema changes from pull request review, the dbt manifest diff or the team's tools, scores their blast radius and plans migrations in approved waves. |
| Governance & Access | 8.5 | 2 | Projects, roles and SSO scope who configures what in Telmai. Data Workers dry-runs each access request and proposes a time-bound grant for the owner to approve. |
| Security & Privacy | 8 | 3 | Hybrid and VPC deployments keep scans in your environment; a schema-only classifier tiers sensitive columns. Data Workers' pull request review flags new columns whose names or annotations look sensitive, and leaves a receipt on every data change. |
| Cost / FinOps | 8 | 1 | Binning keeps bad records out of costly pipeline runs. Data Workers reads AWS Cost Explorer, attributes Snowflake spend to dbt models and drafts compute setting changes for their owner. |
| MLOps & Models | 7.5 | 3 | Telmai validates the data that feeds models and AI agents. Data Workers keeps the data under your models fresh and correct. |
For the category view, read how Data Workers is different from data observability, Bigeye, Anomalo, Sifflet and Soda vs Data Workers, beyond data observability: autonomous resolution and Apache Iceberg for data engineering.
How Telmai and Data Workers work together
Telmai stays on top, where monitors fire, the breaker trips and the team asks Claude about incidents. Spellbook Data Catalog (in preview) is where the data team reviews each proposed change and its rollback. Between them, Data Context Wizard keeps one governed context graph, the Data-Agents Swarm does the work with more than 20 specialist agents, and the Autonomous Data-Conductor runs each fix end to end: detect, diagnose, fix, review, verify, remember.
Telmai to Data Workers. Telmai connects over its MCP server or REST API today: your team's assistant reads incidents, alerts, monitors, lineage and DQ scores side by side with Data Workers, whose own checks rest on the tables and runs its connectors read. PostgreSQL, Step Functions and Slack are native connectors; Iceberg and the Glue Data Catalog, AWS DMS, EMR, Athena and QuickSight connect over their APIs. DMS keeps running its CDC tasks and QuickSight is read by design. When the team hands Data Workers a Telmai incident, a diagnosis starts: diagnose_incident and get_root_cause work the evidence, trace_cross_platform_lineage and blast_radius_analysis map everything downstream, the source change surfaces through change review of the job's pull request, and get_incident_history checks for repeats. remediate proposes the change set: the job change as a diff for the owner to merge, the rule change drafted for the Telmai admin, the rerun queued through Step Functions. monitor_metrics baselines verify on the Data Workers side and the owner confirms the Telmai monitors are green; get_audit_trail reads back every step. The owner closes Telmai's Jira ticket with the receipt link.
Side by side in one client. Telmai documents its MCP server for Claude Desktop, Claude.ai and Cline: one URL per tenant, OAuth through your identity provider for Claude (since 26.3.2, August 13, 2026, with SSO and no client secret) and a short-lived bearer token for Cline. For Data Workers, the client setup guide documents the path: clone the open-source repo and add one start-agent.sh entry per agent. Example cline_mcp_settings.json, with the Telmai entry as Telmai's docs show it:
{
"mcpServers": {
"telmai": {
"url": "https://mcp.env1.telm.ai/<tenant>/mcp",
"type": "streamableHttp",
"disabled": false,
"autoApprove": [],
"headers": { "Authorization": "Bearer <token>" }
},
"dw-incidents": {
"command": "/path/to/dataworkers-claw-community/start-agent.sh",
"args": ["dw-incidents"]
},
"dw-context-catalog": {
"command": "/path/to/dataworkers-claw-community/start-agent.sh",
"args": ["dw-context-catalog"]
}
}
}Ask "why did the premium breaker trip, and what does a fix touch?" and the client calls both: Telmai returns the incident, the failing rule and the binned row count; Data Workers returns the billing release, the column definition, everything downstream and a proposed repair. For a shared endpoint, the Data Workers remote server takes an API key (bearer) or OAuth tokens from your identity provider, such as Okta or Entra ID, verified through JWKS.
One tripped breaker, L0 to L4. The same incident, at each level of the autonomy ladder, set per domain.
- •L0 manual. An engineer digs through DMS batches and the release log, edits the Spark job and restarts the state machine.
- •L1 observe. Data Workers posts the cause and blast radius. Nothing changes.
- •L2 propose. Data Workers proposes the job diff, the rule update and the rerun; nothing runs until the owner approves.
- •L3 act reversibly. For proven change classes, such as rerunning a held batch after a merged fix, Data Workers queues the step, verifies it and records the receipt.
- •L4 autonomous. In a scoped domain with a long clean record, Data Workers runs the repeatable steps end to end; job and rule changes still go to their owners.
For the safety model, read is it safe to let AI agents change production data, how approvals work and autonomy levels L0 to L4. On where data lives: the agents run in your infrastructure and hold the credentials, your data stays in your systems, and the hosted Conductor sees workflow metadata only. For the wider AWS picture, see Data Workers on AWS.
What changes for your team
- •Incidents. A Telmai incident arrives with a diagnosis, a blast radius and a proposed fix, and closes with a receipt.
- •Data quality. Telmai keeps validating at the source; each fix arrives with the rule change drafted for the Telmai admin.
- •Cloud spend. Binning keeps bad batches out of costly runs; compute setting changes go to their owner drafted.
- •Access. A lake access request is dry-run and becomes a scoped, time-boxed grant proposed for the data owner.
- •Audits. Telmai records the incident; Data Workers records what changed, who approved it and how to undo it.
- •Migrations. Each migration wave is planned with its parity checks, tracked, and held at the completion gate for the owner's sign-off.
The on-call changes most: a breaker that trips at 01:19 meets a ready change set, not a blank page. See the data incident response playbook, data quality for AI agents and who owns the agents.
Keep Telmai, or consolidate?
Keep Telmai if you love it; Data Workers works with it from day one. Many teams consolidate once Data Workers runs that slice too.
Teams usually keep Telmai where it validates at the source: raw files and Iceberg tables, binned before any merge, with a breaker in front of every costly run. What they consolidate is the stack around it: a second monitor on the same batch, hand-written rerun runbooks, a spreadsheet of past fixes. Weighing your own fix layer on the Telmai MCP server and a coding agent? Read build it ourselves with Claude Code and MCP servers: the MCP calls are the easy part; the context graph, approvals and rollback are the work. Across the category, see you're on Monte Carlo, you're on Validio, you're on Acceldata and Data Workers integrations.
The case for your CFO
The outcome: Telmai already stops bad data before it reaches the lake. Data Workers turns each tripped breaker from a held pipeline and a morning of tracing into a diagnosis on arrival and a complete fix with one approval, so actuarial and finance read correct numbers on time.
The risk story is plain. Autonomy is set per domain: L1 only reads; L2 changes nothing until a named person approves; L3 applies changes it can undo. Job changes go to their owner as a diff to merge, and Telmai rules stay with the Telmai admin. An unanswered request expires and escalates, never auto-grants. No agent can promote its own work. Every change carries a receipt (cause, diff, approver, what it touched, verification, undo), and an org-wide stop halts all autonomous dispatch. Zero migration: Telmai, Aurora, DMS, Step Functions, the Iceberg tables, Athena and QuickSight stay where they are.
Why now: lakehouses on open table formats now carry the numbers regulators and reinsurers read, and detection is already fast. The slow part is the fix, the rerun and the proof. The first win is read-only: every Telmai incident in one domain gets a cause and a blast radius, while monitors, permissions and change review stay the same. For the numbers, see the ROI of agentic data operations.
The sentence to repeat upstairs: "Telmai stops bad data at the door; Data Workers fixes the cause with one approval and shows us the receipt."
Getting started
Start with a pilot. Pick one domain Telmai already validates, such as policy premium, connect Data Workers to Telmai, Step Functions, the Glue Data Catalog and your Iceberg tables, and run at L1 so every incident gets a cause and a blast radius. Then turn on the first write class at L2, such as rerunning a held batch after a merged fix, with owner approval. The pilot path and plans are on the pricing page, and the pilot is credited in full against the first year.
FAQ
How does Data Workers connect to Telmai? Over Telmai's MCP server or REST API today, reading incidents, alerts, monitors, lineage and DQ scores. Both MCP servers can also run side by side in Claude or Cline.
Telmai already has Data Reliability Agents. Why add Data Workers? Telmai's agents, offered through an early-access waitlist, explain incidents, recommend and apply validation rules, trace lineage and route users inside Telmai, and by Telmai's own policy never write to your source systems. Data Workers adds the cross-system work: the cause in the job or source, the change set with its blast radius, a named approver, the rerun, verification downstream and the receipt.
Does Data Workers change our Telmai monitors or circuit-breaker logic? No. Monitors, rules, binning paths and breaker thresholds stay with your Telmai admins. When a fix needs a rule change, Data Workers drafts it for the admin to apply.
What happens to the rows Telmai binned as invalid? They stay on the invalid path until the owner decides. Valid data in a new shape, as in the example, comes through with the approved fix and a rerun. If they are truly bad, Data Workers proposes the upstream fix and any cleanup for the owner to approve and apply.
We run Telmai in a hybrid deployment so scans stay in our network. Does Data Workers fit that? Yes. The agents run in your infrastructure and hold the credentials, so your data stays in your systems. The hosted Conductor sees workflow metadata only: goals, table names, proposals with diffs, run records, approval handles.
Can Data Workers roll back an Iceberg table? Every change set names its undo before anything runs; for an Iceberg table that is the pre-merge snapshot. The table owner runs the rollback if needed, and Data Workers verifies the result.
Sources
- •Telmai, homepage ("Autonomous ready data for Agentic AI"; "Introducing Telmai's Data Reliability Agents": Orchestration, Validation, Incident Diagnosis, Lineage, Data Insight, Help and Routing agents; "Join Waitlist"; © Telmai 2026), https://www.telm.ai/ (checked Oct 3, 2026)
- •Telmai Docs, release notes (26.3.4, Sep 16, 2026: hybrid deployment model on GCP Dataproc, Azure Databricks and AWS Databricks, Microsoft Fabric connector, Unity Catalog metadata; 26.3.2, Aug 13, 2026: simplified MCP server deployment with SSO; 26.3.0, Jul 1, 2026: AI Agent-powered monitor recommendations), https://docs.telm.ai/telmai/telmai-releases/release-notes (checked Oct 3, 2026)
- •Telmai Docs, Telmai MCP Server (per-tenant URL, OAuth through the tenant identity provider for Claude, bearer token of about one hour for Cline, seven tool groups), https://docs.telm.ai/telmai/integrations/telmai-mcp-server (checked Oct 3, 2026)
- •Telmai Docs, MCP sample prompts ("The assistant can create and change real monitors"; confirm-first guidance), https://docs.telm.ai/telmai/integrations/telmai-mcp-server/mcp-sample-prompts (checked Oct 3, 2026)
- •Telmai Docs, AI Security and Governance (Advisory, Applied and reviewable, Deterministic detection; no writes to source systems), https://docs.telm.ai/telmai/security-and-compliance/ai-security-and-governance (checked Oct 3, 2026)
- •Telmai Docs, Circuit Breaker, https://docs.telm.ai/telmai/remediation/circuit-breaker (checked Oct 3, 2026)
- •Telmai Docs, Configuring Data-Binning (valid and invalid paths on S3, GCS, Azure Blob or Databricks), https://docs.telm.ai/telmai/monitoring-data/monitors-management/configuring-data-binning (checked Oct 3, 2026)
- •Telmai Docs, Jira Integration ("Telmai does not automatically resolve the tickets"), https://docs.telm.ai/telmai/integrations/jira-integration (checked Oct 3, 2026)
- •Telmai Docs, Incidents API, https://docs.telm.ai/telmai/api-reference/incidents-api (checked Oct 3, 2026)
- •Data Workers open-source repository, https://github.com/DataWorkersProject/dataworkers-claw-community (checked Oct 3, 2026)
- •Data Workers client setup guide, https://dataworkers.io/opensource-docs/client-setup/ (checked Oct 3, 2026)