Industry
Industry8 min readBy The Data Workers Team

The agentic data platform for manufacturing data teams

How Data Workers keeps historian, MES and SAP data right as it moves into Databricks or Snowflake: sensor checks, Kafka stream health, S/4HANA moves planned wave by wave with parity checks and traceable genealogy, with a receipt on every change.

For a manufacturing data team, Data Workers, the agentic data platform, keeps plant and enterprise data right as it travels from historians, MES and SAP through Kafka into Databricks or Snowflake: its agents catch the unit change, the stalled stream or the broken mapping, trace what it reaches and propose the fix before the shift review reads it. The same agents plan an S/4HANA move wave by wave with its parity checks and keep genealogy tables traceable, under a named person's approval, with a receipt on every change.

A plant runs on two clocks: the historian writes every second, the ERP closes an order once a shift, and most breaks happen where they join (what is an agentic data platform?).

Key takeaways

  • •OT data that keeps its meaning. Null, row-count and lateness baselines on telemetry tables, with recorded lineage from Kafka topic to table to report.
  • •SAP moves with proof. The migration agent plans each wave with its parity checks, proposes the comparison queries your team runs, tracks the results and holds the completion gate until the owner signs off. SAP connects over its APIs, and nothing writes into SAP.
  • •Traceability you can query on a bad day. Genealogy tables carry freshness SLAs, an authoritative owner and lineage end to end.
  • •Controls mapped to the rule's text. Named approvals, scoped access and a tamper-evident audit trail line up with the EU Data Act, NIS2, the EU AI Act and CMMC; your compliance team decides what satisfies each rule.
  • •Runs where your plants need it. Agents run in your infrastructure on every tier; Enterprise runs on-premise and air-gapped, and sovereign mode blocks external model calls.

The manufacturing data reality

A typical estate runs AVEVA PI System or another historian fed from PLCs and SCADA over OPC UA or MQTT; an MES such as Siemens Opcenter or Rockwell FactoryTalk; SAP ECC or S/4HANA; PLM such as Teamcenter or Windchill; and a QMS. Telemetry and MES events travel over Kafka into Databricks or Snowflake, SAP data increasingly arrives as data products through SAP Business Data Cloud, dbt models it, and the plant manager and the quality engineer look in Power BI.

What hurts:

  • •IT/OT joins. A sensor value means something only with its asset context (line, oven, unit of measure), and a template change or renamed tag breaks that silently.
  • •Stream health. Consumer lag on a telemetry topic means the hourly SPC chart runs on partial data, and nothing errors.
  • •The SAP deadline. SAP provides mainstream maintenance for SAP Business Suite 7 core applications "until end of 2027", with optional extended maintenance to the end of 2030, and every S/4HANA program has to prove open orders, BOMs and material masters came across intact.
  • •Traceability. A recall comes down to one question, which serials used this supplier lot, and the answer is only as good as the genealogy tables.
  • •Edge-to-cloud cost. Telemetry in a cloud warehouse is a spend line. Across industries (not manufacturing alone), dbt Labs' 2026 report found 57% of respondents saw warehouse and compute spend rise, against 36% whose team budgets rose.

Three use cases

1. Sensor and historian data for quality (worked example below). Data Workers sets SLAs on telemetry tables (set_sla), holds the range, unit and lateness metrics the team records to baselines (monitor_metrics), flags shifts against history with get_anomalies, and follows a bad value upstream with get_root_cause and the streaming agent's lineage from topic to table. blast_radius_analysis shows which SPC models, dashboards and lot-release extracts read it. The fix arrives as a diff for the model owner to merge, at L2 propose.

2. An ECC-to-S/4HANA move and the supply chain data around it. SAP data products reach Databricks through SAP Business Data Cloud Connect for Databricks while legacy ECC extracts keep feeding the old models. Data Workers' migration agent plans each wave with parity checks between the tables built from ECC and those built from S/4HANA for open purchase orders, BOMs and material masters, proposes the comparison queries your team runs in the warehouse, tracks the results and holds the completion gate for the owner's sign-off; the cutover decision stays with that named owner. get_quality_score tracks supplier and material master quality through the waves, and define_business_rule records what "on-time in full" means. SAP connects over its APIs or MCP server today, and Data Workers works from the SAP data that lands in Databricks; it never writes into SAP, by design. Start at L1 observe, then L2 propose; you're on SAP Business Data Cloud covers the SAP side in depth.

3. Genealogy and traceability. Siemens Opcenter publishes lot and serial events to Kafka, and Databricks joins them to component lots from SAP and supplier certificates from the QMS in a genealogy table. The streaming agent watches lag and stream health on the MES topic; set_sla and a monitor_metrics baseline keep watch on the table's freshness; trace_cross_platform_lineage shows everything that depends on it; and mark_authoritative, approved by a named owner, makes it the one table the quality team queries. Run it at L1 observe until the team trusts the checks.

Worked example: a unit change in the historian

This is an illustrative example. A Tier 1 automotive supplier runs a cure oven on Line 4. AVEVA PI holds the zone temperatures, the plant's edge integration publishes them to a Kafka topic, a Databricks streaming job lands them in a Delta table, and an hourly Prefect flow builds the dbt model spc_oven_zone_hourly. Power BI shows the SPC charts at the 06:00 shift review, where out-of-control points put lots on hold. The quality-data domain is at L2 propose.

TimeSystemWhat happens
Tue 21:40AVEVA PI SystemA controls engineer updates the oven template in Asset Framework; zone temperatures now report in degrees Celsius
21:41KafkaMessages carry uom: degC; consumers keep up, so nothing errors
22:00PrefectThe hourly flow builds the SPC model on mixed units
22:06Databricks + dbtget_anomalies flags zone temperatures 45% below their history
22:09Databricks + dbtget_root_cause, with streaming lineage from the topic, traces it to rows landed since 21:41 carrying uom: degC
22:12Databricks + dbtData Workers proposes a diff: convert by uom in staging, plus an accepted-values test, with a dry run and rollback path
22:13TeamsThe quality-data on-call gets the proposal and its blast radius: the SPC model, the Power BI report and the lot-release extract
22:31SpellbookThe on-call reviews the dry run and approves
22:33Databricks + dbtThe model owner merges the diff
22:35PrefectData Workers queues the rerun; control limits are back in range
22:41SpellbookReceipt written: diff, approver, time, rollback path
Wed 06:00Power BIThe shift review sees no false out-of-control alarms
Incident timeline across the stack: what Plant data stack, your team and Data Workers each do, step by step

Without the agents, the morning review sees every Line 4 zone out of control and quality holds a night's production. AVEVA PI and Power BI connect over their APIs today; Kafka, Databricks, dbt, Prefect and Microsoft Teams are native integrations among 50+ connectors. Data Workers on Databricks, Data Workers + dbt and you're on Apache Kafka cover the wiring.

Governance and regulation

Data Workers supplies controls and evidence where agents touch data; your controls, people and assessors decide compliance. Not legal advice.

EU Data Act. Regulation (EU) 2023/2854 entered into force on 11 January 2024 and has applied since 12 September 2025. The Commission says it gives users "greater control over the data generated by their connected devices, such as cars, smart TVs, and industrial machinery". Data Workers' side: lineage showing which tables hold connected-product data, and lateness baselines on the datasets you share.

NIS2. Directive (EU) 2022/2555 had to be transposed by 17 October 2024, and the Commission lists "critical product manufacturing" among the sectors in scope (it proposed targeted amendments on 20 January 2026). Data Workers' side: least-privilege access with provision_access (column-level, 90-day default expiry, nothing granted on a review verdict), a hash-chained audit trail you can check with verify_global_hash_chain, and generate_audit_report exports for your security team.

Product traceability. The Ecodesign for Sustainable Products Regulation, Regulation (EU) 2024/1781, has been in force since 18 July 2024 and introduces a Digital Product Passport, "a digital identity card for products, components, and materials". Passport data comes from the BOM, materials and genealogy tables in use case 3, which carry lineage, freshness SLAs and an owner.

EU AI Act. Regulation (EU) 2024/1689, as amended by the Digital Omnibus on AI, Regulation (EU) 2026/1744, moved obligations for high-risk AI in products covered by Annex I, which includes machinery, to 2 August 2028. They include Article 12 logging, Article 14 human oversight with the ability to stop the system, and Article 26(6) log retention of at least six months. Data operations agents are not an Annex III use case on their face; where your counsel applies these articles, per-domain autonomy, named approvals, the org-wide stop and an exportable hash-chained log line up with them (SOX, HIPAA, GDPR and the EU AI Act).

CMMC for defence suppliers. The DFARS rule on assessing contractor implementation of CMMC was published on Sep 10, 2025 and took effect on Nov 10, 2025. For CUI programs, Enterprise runs on-premise or air-gapped, and sovereign mode blocks external model calls so inference stays on a local model (Data Workers in your VPC or air-gapped).

Workforce data. MES records carry operator names and emails; when a change adds such columns, Data Workers' pull request review flags them by name and annotation, the data owner confirms, and masking is proposed for your team to apply (how Data Workers handles PII).

Where it runs and who approves. Your data stays in your systems, and the hosted Autonomous Data-Conductor sees workflow metadata only (where does our data go?). Each domain sits on one ladder, L0 manual, L1 observe, L2 propose, L3 act reversibly, L4 autonomous; approvals go to a named person, an unanswered request expires and escalates, and no agent can promote its own work (how approvals work, is it safe?).

What changes for the manufacturing data team

Six jobs that run on autopilot with Data Workers next to Plant data stack, with a concrete example of each

Six jobs move onto autopilot at the level each owner sets. Data Workers works in your data platform, not on the plant floor: it reads historian and MES data where it lands and has no connection to PLCs or control systems. People still own the decisions: which domains climb the ladder, who approves, and what each business rule means (who owns the agents, will Data Workers replace my data team?).

The alternatives manufacturing teams weigh

Build it yourselves. A strong team can wire a coding agent to MCP servers for Databricks and dbt. The work is everything around it: approvals to the right owner, blast radius, rollback, receipts and context across OT and IT systems (build it ourselves?).

Industrial data platforms. AVEVA describes CONNECT as "an open, vendor-neutral and cloud-based platform" for integrating industrial data, models, applications and AI. Platforms like it are excellent at contextualising OT data; Data Workers runs the operations once that data meets SAP, dbt and BI.

ERP and platform-native agents. SAP announced Joule Agents in Business Data Cloud on May 13, 2026; Snowflake made its managed MCP server generally available on Nov 4, 2025; Databricks made Genie One generally available on Jun 16, 2026. Each is strong inside its own platform, by design. A plant's data crosses the historian, Kafka, the warehouse, SAP and BI, and changing it across all of them takes blast radius, approvals, rollback and receipts in one place: that's the product Data Workers is.

The financial services and retail and e-commerce pages map the same controls to their rules, and AI for data infrastructure in manufacturing covers the topic from the infrastructure side.

The case for your CFO

The outcome is fewer false holds and faster answers. A unit change that reaches the SPC chart holds a line's output; a broken genealogy join widens a recall; a cutover without parity proof becomes a quarter of reconciliation.

The risk story: at L2 propose every change waits for a named owner, nothing writes into SAP or the control systems, and every change carries a receipt with the diff, the approver and the rollback path. What stays the same: your historian, MES, SAP, warehouse and dashboards, with nothing migrated.

Why now: SAP's mainstream maintenance for Business Suite 7 ends in 2027, the EU Data Act has applied since September 2025, and product passport data is coming. The evidence has to exist anyway.

The first win: sensor checks and incident diagnosis on one line's telemetry and its SPC models.

Start with a pilot ($7,500 one-time, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing, the ROI calculator and the ROI of agentic data operations. The sentence for upstairs: "Our plant and SAP data now has a crew that catches breaks before the shift review, and every fix has a receipt."

FAQ

Does Data Workers connect to AVEVA PI and SAP? Yes. Data Workers connects to AVEVA PI, SAP and Siemens Opcenter over their APIs today, and works natively with the places their data lands: Kafka, Databricks and Unity Catalog, Snowflake, dbt, and orchestrators such as Airflow, Dagster and Prefect, among 50+ connectors (Data Workers integrations).

Can Data Workers watch our Kafka streams? Yes. Kafka, Kafka Connect and Schema Registry are native. The streaming agent tracks consumer lag and stream health per topic and keeps lineage from topic to table, so a lagging telemetry topic shows up before the hourly model runs on partial data.

We're a defence supplier. Can it run inside our CUI enclave? Enterprise runs on-premise and air-gapped, the agents hold your credentials inside your environment, and sovereign mode blocks external model calls. Your assessor decides how that fits your CMMC level; Data Workers supplies the access controls and the audit trail.

How does it help with an S/4HANA migration? The migration agent plans each wave with its parity checks, proposes the comparison queries your team runs, tracks the results and holds the completion gate; a named owner signs off and makes the cutover call. Hive and Glue to Unity Catalog with parity checks shows the same method on a lakehouse move.

Sources

  • •SAP Support, Maintenance strategy for SAP S/4HANA and SAP Business Suite 7 ("mainstream maintenance until end of 2027 for SAP Business Suite 7 core applications"; extended maintenance "from beginning of 2028 until end of 2030"): https://support.sap.com/en/release-upgrade-maintenance/maintenance-information/maintenance-strategy/s4hana-business-suite7.html (checked Oct 2, 2026)
  • •European Commission, Data Act (in force 11 Jan 2024, applicable from 12 Sep 2025): https://digital-strategy.ec.europa.eu/en/policies/data-act (checked Oct 2, 2026)
  • •European Commission, NIS2 Directive (transposition by 17 Oct 2024; "critical product manufacturing"; amendments proposed 20 Jan 2026): https://digital-strategy.ec.europa.eu/en/policies/nis2-directive (checked Oct 2, 2026)
  • •European Commission, Ecodesign for Sustainable Products Regulation, Regulation (EU) 2024/1781 (in force since 18 Jul 2024; Digital Product Passport): https://environment.ec.europa.eu/strategy/circular-economy/ecodesign-sustainable-products-regulation_en (checked Oct 2, 2026)
  • •EU AI Act, Regulation (EU) 2024/1689: https://eur-lex.europa.eu/eli/reg/2024/1689/oj ; Digital Omnibus on AI, Regulation (EU) 2026/1744 (OJ 24.7.2026): https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202601744 (checked Oct 2, 2026)
  • •Federal Register, DFARS: Assessing Contractor Implementation of Cybersecurity Requirements (DFARS Case 2019-D041), published Sep 10, 2025, effective Nov 10, 2025: https://www.federalregister.gov/documents/2025/09/10/2025-17359/defense-federal-acquisition-regulation-supplement-assessing-contractor-implementation-of (checked Oct 2, 2026)
  • •dbt Labs, 2026 State of Analytics Engineering Report (cross-industry; 57% increased warehouse and compute spend, 36% increased team budgets): https://www.getdbt.com/resources/state-of-analytics-engineering-2026 (checked Oct 2, 2026)
  • •AVEVA, AVEVA PI System: https://www.aveva.com/en/products/aveva-pi-system/ (checked Oct 2, 2026)
  • •AVEVA, CONNECT industrial intelligence platform: https://www.aveva.com/en/solutions/connect/ (checked Oct 2, 2026)
  • •Databricks, Connect to Apache Kafka (Structured Streaming, updated Sep 11, 2026): https://docs.databricks.com/aws/en/connect/streaming/kafka (checked Oct 2, 2026)
  • •SAP News, SAP Business Data Cloud at Sapphire (May 13, 2026; Joule Agents): https://news.sap.com/2026/05/sap-bdc-accelerate-autonomous-enterprise/ (checked Oct 2, 2026)
  • •SAP, Business Data Cloud open data ecosystem (SAP Business Data Cloud Connect for Databricks): https://www.sap.com/products/data-cloud/open-data-ecosystem.html (checked Oct 2, 2026)
  • •Snowflake, Snowflake-managed MCP server GA release note (Nov 4, 2025): https://docs.snowflake.com/en/release-notes/2025/other/2025-11-04-cortex-agents-mcp (checked Oct 2, 2026)
  • •Databricks, Genie One launch press release (Jun 16, 2026): https://www.databricks.com/company/newsroom/press-releases/databricks-launches-genie-one-all-new-agentic-coworker-every-team (checked Oct 2, 2026)
  • •Data Workers security page (last updated Sep 10, 2026): https://dataworkers.io/security/ ; pricing: https://dataworkers.io/pricing/ (checked Oct 2, 2026)
  • •Data Workers open-source repository, tool registrations: https://github.com/DataWorkersProject/dataworkers-claw-community (checked Oct 2, 2026)