Entity resolution teams: ship every re-match as a tested release

Key takeaways

• •Who this is for. The team at a data business that matches, merges and de-duplicates records from many suppliers into one profile per company, person, place or product, and so owns the unified profile the company sells. It includes the master data (MDM) and golden record teams inside enterprises that productise or resell data.
• •The problem today. Match rules drift as suppliers change their feeds, survivorship rules (which source wins for each attribute) live in someone's head, every re-match run is a release nobody can fully test, customers dispute merges and the answer takes days, and consent provenance has to be reconstructed attribute by attribute.
• •What changes. The matching you already run in your own warehouse gets agents around it. Match rate, duplicate profiles and merge counts are watched for every supplier every day. A re-match is checked against the last release before it ships. A disputed merge is traced to the rule and the source records in minutes. Every survivorship rule is written down with an owner and a history.
• •What stays with your people. Uncertain matches and every survivorship rule change go to a named data steward. Agents prepare the evidence and the diff; a person decides.
• •The result. A false merge shipped to a customer is a product defect and a split profile is lost coverage. Agents catch both before the release, so the team spends its week widening coverage instead of explaining merges.

The PRD for agentic entity resolution and unified profiles teams

This is the requirements document we would write with you for your team. Everything below expands on it.

What going fully agentic and autonomous means for entity resolution and unified profiles teams

• •Put your data pipeline on autopilot. A supplier's identifier, name or address format change is caught and mapped the day it happens, so a Sunday file does not become a wave of duplicate profiles on Monday.
• •Run the data back office without the busywork. The work that fills the week today moves to agents: sampling merges by hand before a release, digging through run logs to answer a customer dispute, rewriting the survivorship documentation after the last change and chasing consent provenance attribute by attribute.
• •Scale data without scaling headcount. Ten suppliers or a hundred, each one gets the same release checks, monitoring and documentation, so the number of sources feeding a profile grows without the stewardship queue growing with it.
• •Get the plumbing off your plate. An entity resolution team that improves coverage and accuracy of the profile, rather than defending last night's run.

Who this is for

The team that turns many overlapping records into one profile that customers pay for. At a B2B company and contact data provider, it resolves firms, sites and people across registries, web-derived sources and licensed suppliers. At an identity graph or people data vendor, it links records for the same person across suppliers and devices while keeping consent attached. At an alternative data vendor, it maps merchants, tickers, locations and brands in panels and transaction feeds to a clean entity reference. At an enterprise that resells or productises data, such as a bank, telco or retailer packaging insights for outside buyers, the master data team owns the golden record behind the product.

The titles: Head of Entity Resolution, Director of Master Data, Identity Resolution Lead, Data Steward, Data Product Manager (Entity Data), and the data engineers and analysts who report to them. These leaders are measured on the commercial metrics of the product the company sells: coverage, accuracy, freshness, match rate, false merges, false splits, delivery and sell-through. Their output is the spine every other data team in the family hangs on. Data acquisition and sourcing feeds it, company and person data is the product that sits on it, data delivery and feeds ships it, and coverage, accuracy and evaluation measures it. Buyers in financial services rely on it for counterparty and onboarding data, where a wrong merge is a wrong risk decision. Teams on the buy side face a mirror image in market and reference data, and the same questions come up in geospatial and marketplace data, people and labor-market data and Customer 360 and CDP data. All of them are gathered on the data businesses page.

The product you ship

• •The unified profile itself: one governed profile per company, person, place or product, with a stable identifier that does not change when a supplier does.
• •The cross-reference: the link from every supplier record to the profile it belongs to, so a customer can bring their own identifiers and get yours back.
• •Survivorship and attribute provenance: for each attribute on the profile, which supplier it came from, when, and under what terms and consent.
• •Match quality measures: match rate by supplier and entity type, duplicate profile counts, and the false merge and false split rates your stewards measure by sampling.
• •A steward queue: the uncertain matches, the disputed merges and the rule changes waiting for a decision, with the evidence attached.
• •Release notes for customers: what changed between profile versions, including merges, splits and retired identifiers, so a customer's own join does not silently break.

Where the week goes

• •Match rules drift as suppliers change. A new identifier format, a renamed country field, a supplier that starts sending legal names where it sent trading names. Nothing fails; the match rate falls and duplicate profiles pile up.
• •Every re-match run is a release. It changes profiles customers already use. Testing it means sampling merges by hand, which covers a sliver of the output, so most releases ship on trust.
• •Survivorship rules live in someone's head. Which supplier wins for address, headcount or phone is in a stored procedure, a wiki page nobody trusts or the memory of the engineer who built it.
• •Customers dispute merges. The ticket says two companies were combined, or one was split. Answering means finding the run, the rule and the source records, usually across three systems.
• •A false merge is a product defect, a false split is lost coverage. Both are found by customers first, because the team's own checks are periodic samples.
• •PII and consent provenance travel per attribute. A profile blends fields from suppliers with different notices and permitted uses. When a person asks for their data to be removed, every supplier record merged into the profile has to be found.
• •Stewardship is a queue that only grows. Uncertain matches arrive faster than people can decide them, and the cases that need judgment sit behind the ones that do not.

Add it up and upkeep is most of the team's week. Here is how a typical week splits today, and once agents own the back office:

Time spent on new data products, which for this team means wider coverage and new entity types, moves from 14% of the week to 48%. Most of the rest becomes review: people steer and approve agent work instead of doing it by hand.

Then and now

These are the impact metrics we move for entity resolution and unified profiles teams:

• •Disputed merge traced to its rule and source records: from about two days to ~15 min
• •Match-rate drop traced to its cause: from about three days to under an hour
• •Re-match checked before release: from about two days to about an hour

Across the routine work in this domain:

The biggest change is tracing a disputed merge: from about two days to ~15 min. Work that waited in queues finishes in minutes or hours, because agents detect, trace and verify in one pass.

Use cases, in depth

Every re-match run ships as a tested release

Today. The nightly run rebuilds profiles from every supplier. A rule edit, a new supplier or a changed feed alters thousands of profiles, and the team checks a few dozen by eye. The release goes out and the first complaint is the test.

With Data Workers. The Pipeline Building agent builds the change to the matching pipeline you already run in your warehouse as a reviewed diff, with tests, and validates it before it deploys. Quality Monitoring compares the candidate run with the last release: match rate by supplier, merges, splits and duplicate counts. The Data Context & Catalog agent lists the profiles and the customer feeds that would change. The release goes ahead when the comparison is clean, and goes to the data steward with the evidence when it is not.

What changes. Every re-match run arrives with the evidence that it did not make the product worse.

A customer disputes a merge

Today. A customer reports that two of their accounts were merged into one profile. Someone finds the night it happened, reads the run output, works out which rule and which supplier records were involved, and replies after two days. The same rule has merged other profiles, and nobody knows how many.

With Data Workers. The Incident Debugging agent opens the dispute as an incident and traces it to the run, the rule and the source records that joined. The Data Context & Catalog agent shows where each attribute on the profile came from, with lineage across the warehouse and the transformation layer. The same agent lists every other profile the rule touched, so one dispute becomes one fix. The data steward gets the evidence, a proposed split and the list of other affected profiles.

What changes. The customer gets an answer the same day, and the defect is fixed everywhere it occurred, not only where it was reported.

Survivorship rules written down, with an owner

Today. Which supplier wins for each attribute is decided in code that one or two people understand. When a supplier is added or dropped, nobody can say which fields change owner. When a customer asks why a value is on the profile, the answer is an investigation.

With Data Workers. The Data Context & Catalog agent imports the rules from the code, documents and conversations where they already live, records each as a governed business rule with an owner, and keeps it current. Any proposed change to a rule arrives at the data steward as a diff with the profiles and attributes it would change. Spellbook Data Catalog gives each attribute a page in plain English: where it comes from and why that source wins.

What changes. A survivorship rule is no longer knowledge that leaves with a person.

Measure the product, not just the pipeline

Entity resolution leaders are measured on what the profile sells, not on how many jobs ran. The Autopilot dashboard shows how much of each routine's upkeep runs on its own and at what token cost. Alongside it, the team can track what leadership and customers care about:

• •Coverage: the share of the entities in your market that have a profile, and the lift from each new supplier.
• •Accuracy: false merges and false splits, as your stewards define and measure them, and the share of attributes that are populated and agree with a reference.
• •Match rate: the share of supplier records that resolve to a profile, by supplier and entity type, and how it moves release to release.
• •Freshness: time from supplier delivery to a changed profile in the product, against what you promise.
• •Delivery and sell-through: which profile versions reached customer feeds and which sold datasets depend on a supplier that is slipping, which is where data delivery and coverage and accuracy evaluation pick up.

For enterprise master data and golden record teams

An enterprise MDM team inside a bank, telco, retailer or manufacturer usually starts with an internal job: one golden record for customers, products or suppliers across systems. The same team often ends up productising it, selling insights, panels or reference data to outside buyers. That changes the standard. An internal duplicate is an annoyance; a false merge in a dataset sold to customers is a defect they can dispute and a contract question. The same agents apply. Data Workers works alongside your MDM tool or the matching in your warehouse, adds release checks and monitoring around it, documents survivorship, and keeps the steward queue short. For the customer-facing profile side of this problem, see Customer 360 and CDP data. The step from "master data for ourselves" to "a data product we sell" is the one where release evidence becomes a commercial necessity.

Agents working together: a supplier changes its identifier format on a Sunday night

Every entity resolution team has had this Monday. A supplier's weekly file arrives on time with every column in place, so nothing fails. The values in one identifier column are in a new format. The overnight run cannot link those records to existing profiles, creates thousands of new ones and the product ships with duplicates, and a customer is the first to notice. This is the same Sunday night run by Data Workers.

The setup

The team resolves companies from fourteen supplier feeds into one profile per company, built nightly in Snowflake with dbt under Airflow and delivered to customers through a data share and an API on Monday morning. Autonomy in Autopilot:

• •Monitoring, reruns and catalog upkeep: L2 today; a candidate for L3 on evidence.
• •Mapping changes for known suppliers: L2, held for the data steward.
• •Survivorship rule changes, uncertain matches and unmerge requests: L2, held for the data steward.

The team scheduled one mission earlier: "Compare every nightly run with the last release before it publishes."

02:10: the file arrives

The supplier drops its Sunday file. The layout is unchanged.

02:40: flag it

Quality Monitoring agent compares the nightly run with the last release. The match rate for that supplier has fallen from 94% to 71%, and the count of new profiles created is about three times normal. The other thirteen suppliers are in range. No schema alert fires, because the schema is intact; the quality check does.

03:05: find the cause

Incident Debugging agent traces the change to the identifier column in that supplier's file, which changed format with this delivery. Nothing else in the pipeline or the rules changed.

03:20: see who is affected

Data Context & Catalog agent follows lineage from the supplier's records to the profiles they created. About 6,200 new profiles are duplicates of existing ones, and they would reach three customer feeds and the data share.

03:35: draft the fix

Pipeline Building agent drafts the mapping change in the team's own pipeline for the new format, as a reviewed diff with tests, and validates it against last week's file, which it reproduces unchanged.

04:10: one decision

Autonomous Data-Conductor holds only that supplier's records out of the Monday release, lets the other thirteen publish, and sends the data steward one decision with the evidence and the options: approve the mapping fix and re-run that supplier, or publish with last week's profiles for those records.

06:15: approved

The steward approves the mapping fix and resolves the 31 uncertain matches the re-run left for a person. Conductor applies the change.

06:40: verified

Quality Monitoring agent confirms the supplier's match rate is back at 94% and new profile creation is back in range. The held records publish.

07:00: customers get the feed

Customers receive complete profiles on schedule, with no duplicates and no ticket.

What stays behind

• •A record of what a normal nightly run looks like for each supplier, so the next drift is compared with the same picture.
• •A receipt for every change: who proposed it, who approved it and how to roll it back.
• •An updated Spellbook page for the supplier and the identifier, with the incident attached.
• •A reusable mission. Next time a supplier changes a format, the same routine catches it the same way.

Minute by minute

This is an illustrative example. The profile is built in Snowflake with dbt under Airflow and delivered on Monday morning. Matching changes and unmerge requests are at L2.

The agents on the job

Data Context Wizard holds one governed definition of each measure the company sells, such as match rate, false merge rate and coverage, so a release report, a sales deck and a customer data sheet quote the same number. Autonomous Data-Conductor verifies the outcomes that matter before a profile release is delivered. Spellbook Data Catalog gives each entity type, attribute and supplier a page in plain English: what it is, where it comes from, what you may do with it and who owns it.

What goes on autopilot

Once a routine has earned it, this work runs on its own, and each item removes a problem your team deals with today:

• •Every nightly run is compared with the last release before it publishes, so a sag in match rate or a jump in duplicate profiles is stopped at your door, not found by a customer.
• •Match rate, duplicates and merge counts are watched per supplier every day, so drift shows up as one supplier's trend line instead of a customer ticket.
• •Disputed merges are traced to the run, rule and source records, so the steward starts with the evidence rather than a hunt.
• •Rule documentation, attribute provenance and lineage are kept current, so the answer to "why is this value on the profile" is always on the page.
• •Stale match tables and old profile copies are flagged for retirement, so you stop paying to store versions no customer reads.

Where your people stay in charge

Agents never get blanket access. Each routine has its own autonomy level, set by you, and it only rises when the receipts show the agents are reliable. Everything starts at L1, observing only; reversible upkeep such as reruns, documentation and monitoring is usually first to run on its own.

• •Uncertain matches go to a named data steward. Agents bring the records, the evidence and the likely consequences of each choice; a person decides.
• •Every survivorship rule change goes to a named data steward as a diff showing the profiles and attributes it would change.
• •Unmerge, split and delete requests on delivered profiles go to a named approver, because they change what customers already hold.
• •PII and consent provenance decisions go to a named approver. If a source carries personal data without a clear basis, the agents hold it out of the shared layer and ask.

Once a deployment has matured, here is how that ladder plays out across task classes:

Catalog and docs upkeep closes autonomously 88% of the time. For survivorship rule changes, autonomous closure is 0% by design: every change is proposed with a full diff and approved by a named steward.

What happens when an agent is wrong

• •At L2 propose, nothing changes until a named person approves. The proposal shows the full diff, the profiles it touches and how to roll it back. Unanswered requests expire and escalate.
• •L3 and L4 are where a domain can go next, on its own review record: acting on its own, with a rollback point on every change. Only the domain's owner raises its level, and if people start reverting the agents' changes, the domain pauses itself.
• •No agent approves or promotes its own work. Autonomy only grows on evidence, domain by domain, and repeated reverts pause the domain.
• •One org-wide stop halts all autonomous dispatch, and every action lands in a tamper-evident audit trail you can export with generate_audit_report.

How it fits your stack

Data Workers connects to what you already run: supplier SFTP, S3 and API drops; Snowflake or Databricks; dbt; Airflow or Dagster; the matching and survivorship logic in your warehouse or MDM tool; a catalog; and the delivery layer your customers buy through. Each agent is an MCP server, so the same agents work inside Claude Code, Cursor or Codex and in Slack or Teams through Spellbook. Over 50 connectors are available. Nothing migrates, and the matching stays yours, in your own warehouse.

• •Runs in your infrastructure with your credentials and your model key. The hosted Autonomous Data-Conductor sees workflow metadata only, never rows. That matters when the rows are licensed supplier data and personal records.
• •Uses the access you already govern. Agents act under scoped service identities and your existing roles; access to licensed or personal data goes through check_policy and is logged.
• •Fits a per-supplier data contract. Layout and freshness expectations for each supplier can be written down in an open format such as the Open Data Contract Standard, and agents check each delivery against it.
• •Proposes changes as diffs in your dbt project, orchestrator and warehouse, with blast radius, tests and a rollback point, and applies them only at the autonomy level you set for that routine.
• •Open-source core. The Apache-2.0 community edition ships 11 agents and 160+ MCP tools, so your engineers can read exactly what the agents call.

How the pieces behave in practice is covered in Inside the Schema Evolution Agent, which handles supplier layout changes, and Inside the Quality Monitoring Agent, which handles the release comparison. For the catalog side, see the complete guide to data catalogs, which covers entity resolution as part of the metadata picture.

Governance and regulation

A unified profile is a claim about the world, and for people it is personal data. Under the GDPR, personal data must be accurate and kept up to date (Article 5(1)(d)), a person can ask for inaccurate data to be corrected (Article 16) or erased (Article 17), and a company that obtained the data from a source other than the individual has notice obligations to that person (Article 14). In a merged profile, each of those obligations reaches back to every supplier record that was combined into it. When a customer, a regulator or a data subject asks what a profile is based on, the answer has to come from the pipeline, not from a search through run logs.

Data Workers keeps attribute-level provenance attached to each profile, flags personal data with scan_pii before it reaches a shared layer, holds uses that the source terms do not clearly allow for a named approver, and records every change in a tamper-evident audit trail. See how Data Workers handles PII and AI data stewardship.

Data Workers doesn't make you compliant; your controls, people and assessors do. It supplies controls and evidence wherever agents touch data. Not legal advice.

The alternatives entity resolution teams weigh

Build it yourselves. A strong team can wire a coding agent to MCP servers for the warehouse and dbt and generate pipeline changes quickly. The work is everything around it: routing each decision to the right steward, blast radius across delivered profiles, rollback, receipts, and a shared record of what every rule and every supplier's terms allow.

Master data management suites and identity resolution tools. These run the matching and the stewardship workflow, and they are good at it. Data Workers does not replace them. It sits around them, comparing each release with the last, tracing disputes, watching supplier drift and keeping the documentation current. If you are on one, see how this works alongside Ataccama.

Observability, catalogs and point tools. Observability alerts, catalogs document and quality tools test. Data Workers closes the gap between the alert and the fix, and keeps the rule documentation and provenance current as a by-product.

Platform-native agents. Warehouse vendors' agents work well inside their platform. Profile production crosses the supplier's drop, the orchestrator, the warehouse and the delivery layer, and Data Workers works across all of them.

In this space specifically. Among tools that run matching and master data, Data Workers is the operations layer for the release: it checks the run, traces the dispute and holds the change for the steward, whichever tool does the matching.

The case for your CTO

What this replaces. Releases that ship on a hand sample, days of log archaeology per dispute, survivorship rules that live in one engineer's head, and consent provenance rebuilt on request.

What your team gets back. An entity resolution team that widens coverage and raises accuracy, and a data product whose profiles come with evidence.

Integration effort. Connecting is configuration, not a migration. The agents attach over MCP to the supplier drops, warehouse, orchestrator and dbt project you already run, under the roles you already govern, inside your own infrastructure. Your engineers can read every tool the agents call in the open-source core.

Risk and control. Autonomy is set per routine. Everything starts at L1, observing only; reversible work moves up only when the receipts justify it; uncertain matches, survivorship rules and unmerge requests stay with a named steward; and one org-wide stop halts all autonomous work.

Why now. Customers compare data vendors on coverage and accuracy, and AI agents now consume profiles directly, so a false merge reaches more places faster and gets acted on without a human in between.

The first win. Release checks on the nightly re-match and dispute tracing for one entity type, at L2. The pilot below lays out the scope, what you see each week and the cost.

The climb: from assisted to autonomous to agentic

Our thesis is that the data platform climbs the way analytics did, in stages, and that the hard part of data work was never knowing what to do but doing it. For entity resolution and unified profiles teams, the climb looks like this:

• •Assisted. Agents work alongside your team. They read every supplier feed and every release, show what needs doing and propose every fix with a receipt; uncertain matches and survivorship changes always go to a named steward.
• •Autonomous, under your governance. Routine work runs on its own: every run is compared with the last release, match rate and duplicates are watched per supplier, and disputes are traced to their cause. Your team governs the agents instead of doing the upkeep.
• •The agentic data business. An entity resolution team that improves the profile instead of defending the last run. People set direction, and the platform carries it out.

You choose the altitude, every step is visible and reversible, and the pilot below is the first rung.

The pilot: how entity resolution teams start

The pilot is how your team proves the value on its own data before committing.

• •What we connect. Your supplier drops, Snowflake or Databricks, dbt, Airflow or Dagster, and the delivery layer, over MCP, inside your infrastructure. Nothing migrates.
• •Scope. Release checks on the nightly re-match plus dispute tracing for one entity type, at L2.
• •First week. Agents run at L1 across the scoped routines. They profile every supplier delivery, map lineage from supplier records to profiles and show the checks they would have run and the disputes they would have traced, with receipts, so your team sees the value before anything changes.
• •By week two. Run-to-run comparison, per-supplier monitoring and documentation upkeep start running on reversible work. Matching changes, survivorship rules and anything touching personal data stay at L2 with a named steward.
• •At day 30. We review the impact metrics against your baseline:

• •Disputed merge traced to its rule and source records: from about two days to ~15 min
• •Match-rate drop traced to its cause: from about three days to under an hour
• •Re-match checked before release: from about two days to about an hour
• •the share of catalog and docs upkeep closed autonomously, targeting 88% as the deployment matures

• •What you decide. Whether to expand to the next entity type. The pilot fee is credited in full against the first year.
• •Cost. Start with a pilot ($7,500 one-time, a 6 to 12 week pilot, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.

The pilot is the start of a longer curve:

By month 12, more than 42% of agent actions run fully autonomously and manual work falls below 10%, while uncertain matches and survivorship rules stay with a named steward.

FAQ

Do agents replace our matching? No. The matching stays what you run today in your warehouse or MDM tool. Agents build, test and monitor around it.

Who decides an uncertain match? A named data steward. Agents bring the records and the evidence; a person decides.

Can agents change a survivorship rule? They can propose one, with the profiles and attributes it would change. A named steward approves every change.

Can it help with a customer's merge dispute? Yes. It traces the profile to the run, the rule and the source records, and lists other profiles the same rule touched. The decision to split goes to a named approver.

How does it handle personal data in supplier records? It flags personal data fields before they reach a shared layer and keeps the provenance you record with each source. Whether a use is allowed goes to a named approver.

Where do the agents run? In your infrastructure, with your credentials and model key. The hosted Conductor sees workflow metadata only.

Do we have to migrate anything? No. Your warehouse, dbt project, orchestrator, MDM tool, catalog and delivery layer stay where they are.

Sources

• •Data Workers pricing: https://dataworkers.io/pricing/
• •European Union, General Data Protection Regulation (Regulation (EU) 2016/679), including Article 5(1)(d) on accuracy, Article 14 on information where personal data have not been obtained from the data subject, Article 16 on rectification and Article 17 on erasure: https://eur-lex.europa.eu/eli/reg/2016/679/oj
• •Bitol, Open Data Contract Standard: https://bitol-io.github.io/open-data-contract-standard/