Agentic data acquisition: a guide for foundation model data teams
This guide explains how data acquisition teams adopt agents for their data back office. It covers the shift underway, the use cases that matter most, charts of the impact, how the agents work together, and a month-by-month path to autonomy.
Key takeaways
- •Who this is for. The teams that feed foundation models: data acquisition and partnerships teams that source, vet and license real-world data assets; training data engineering teams that turn those assets into tokens; and research data teams that measure which data actually improves the model.
- •The problem today. Two weeks of manual profiling per seller sample, overlap discovered after purchase and month-long processing queues.
- •What changes. Seller samples are profiled for overlap, contamination, PII and license flags before negotiation, so you stop paying for data you already own. Signed assets are processed into training-ready tokens as a mission, so researchers train on new data in days.
- •What stays with your people. Changes to released training mixes and license scope: these carry legal and model risk, so a named lead approves.
- •The result. An acquisition team that negotiates on unique usable tokens and gets new assets into training in days.
The PRD for agentic data acquisition and foundation model data teams
This is the requirements document we would write with you for your team. Everything below expands on it.
| Who | Data acquisition leads, training data engineering, data partnerships and data-to-model optimization teams at frontier labs, model builders and AI data companies |
| The job | Every purchase priced on unique usable tokens. |
| Environment | Seller data rooms and buckets; object storage; Spark or Ray; dedup and filter pipelines; training run trackers; eval stores. |
| Constraints | Changes to released training mixes and license scope always go to a named approver. Acquired data carries license, privacy and provenance obligations. |
| Success is measured by | Asset traced to eval delta: from about 2 days to seconds; License change enforced: from about 2 weeks to ~1 h; Duplicate corpus copies retired: from about a month to hours; sample profiling closed autonomously 86% of the time. |
| Owned by | A Data Acquisition Lead, with domain owners as named approvers. |
| Pilot scope | Vet the next three seller samples. |
What going fully agentic and autonomous means for data acquisition and foundation model data teams
- •Put your data pipeline on autopilot. Seller samples are profiled for overlap, contamination, PII and license flags before negotiation, so you stop paying for data you already own.
- •Run the data back office without the busywork. The work that fills the week today moves to agents: two weeks of manual profiling per seller sample, overlap discovered after purchase and month-long processing queues.
- •Scale data without scaling headcount. Signed assets are processed into training-ready tokens as a mission and cross-asset duplicates are removed and storage tiered, so the data you manage grows without the team having to.
- •Get the plumbing off your plate. An acquisition team that negotiates on unique usable tokens and gets new assets into training in days.
Who this is for
The teams that feed foundation models: data acquisition and partnerships teams that source, vet and license real-world data assets; training data engineering teams that turn those assets into tokens; and research data teams that measure which data actually improves the model. They sit at frontier labs, at companies training their own models, and at the applied data labs that supply them. The roles are new and growing fast: Data Acquisition Lead, Head of Data Partnerships, Training Data Engineer, Pretraining Data Lead and Data Strategy for Model Development.
The shift
The next generation of models needs an order of magnitude more high-quality data than the last, and the open web has been mined. The frontier is now proprietary enterprise data, licensed content, expert data and synthetic data, acquired deal by deal. That changes the job. Acquisition teams negotiate seven-figure data purchases, and the value of each deal depends on questions that are hard to answer quickly: how much of this asset is new, how clean is it, what are we allowed to do with it, and did it make the model better?
Today those questions are answered by hand, after the money is spent. Our view: the path from a seller's sample to a measured model gain should be one governed pipeline run by agents. Vet assets before the term sheet, land and process them in days, enforce license scope on every training run, and tie every asset to the eval deltas it produced, so the next acquisition is priced on evidence.
The product you ship
- •Acquired data assets, vetted, licensed and packaged, with provenance and usage scope recorded.
- •Training-ready datasets: normalized, filtered, deduplicated, decontaminated against eval sets and tokenized.
- •Data mixtures for pre-training, mid-training and post-training, versioned.
- •Ablation results that connect a dataset to the model gain it produced.
- •Acquisition ROI reporting: cost per unique usable token, by asset and partner, for leadership.
Where the week goes
- •Vetting is manual. Each seller's sample is pulled, profiled and read by people before a negotiation can start.
- •Nobody knows the overlap until after the purchase. Duplicate content across assets and with the existing corpus is found after signing.
- •Formats are endless. Enterprise exports, document dumps, logs, transcripts and code arrive in hundreds of shapes.
- •License terms live in contracts, not in the data. Usage scope, field restrictions and retention limits are tracked in spreadsheets.
- •Ablation bookkeeping is fragile. Which dataset version went into which run, and what changed in the evals, is reconstructed by hand.
- •Storage and processing costs grow faster than the corpus. Every experiment copies data.
Add it up and upkeep is most of the team's week. Here is how a typical week splits today, and once agents own the back office:
Time spent on new data products moves from 14% of the week to 50%. Most of the rest becomes review: people steer and approve agent work instead of doing it by hand.
Then and now
| Work | Today | With Data Workers |
|---|---|---|
| Vetting a seller's sample | One to two weeks of manual profiling | A vetting report in hours: unique tokens, quality, PII, overlap, license flags |
| Overlap with the existing corpus | Found after the purchase | Measured before the term sheet |
| Signed asset to training-ready tokens | Weeks of format work and filtering | Days, as a processing mission |
| License scope on training runs | Tracked in contracts | Enforced on every dataset and run |
| Which data improved the model | Reconstructed from notes | Lineage from asset to dataset version to run to eval delta |
| Cost of 10x more data | Grows faster than the corpus | Deduplicated, tiered and right-sized continuously |
These are the impact metrics we move for data acquisition and foundation model data teams:
- •Asset traced to eval delta: from about 2 days to seconds
- •License change enforced: from about 2 weeks to ~1 h
- •Duplicate corpus copies retired: from about a month to hours
Across the routine work in this domain:
The biggest change is License change enforced: from about 2 weeks to ~1 h. Work that waited in queues finishes in minutes or hours, because agents detect, fix and verify in one pass.
Use cases, in depth
Vetting a 40 TB asset before the term sheet
Today. The seller shares a sample. Engineers pull it, write one-off scripts to profile it, researchers read examples, legal reviews the license draft. Two weeks later the team has an opinion, but not the number that matters: how many genuinely new, usable tokens are in it.
With Data Workers. Connectors pull the sample from the seller's data room or bucket. Data Ingestion profiles formats, volumes and languages. Data Security scans for PII and sensitive content. Quality Monitoring measures quality signals, near-duplicate overlap with your existing corpus and contamination with your eval sets. Data Context Wizard records the proposed license terms against the asset. Autonomous Data-Conductor assembles a vetting report: estimated unique usable tokens, quality distribution, PII exposure and license flags.
What changes. The acquisition lead negotiates on unique usable tokens, not raw terabytes, and walks away from assets that mostly duplicate what the lab already owns.
From signed deal to training-ready tokens in days
Today. Once the deal closes, the asset waits in a queue for format conversion, cleaning, filtering, dedup and decontamination, while researchers wait for it.
With Data Workers. Processing runs as an Autopilot mission: Data Ingestion lands the full asset, Schema Evolution normalizes its formats, Pipeline Building runs the versioned filter, dedup and decontamination pipelines with tests, Data Access & Governance applies the license scope to every output, and Data Context & Catalog publishes the dataset with provenance in Spellbook.
What changes. Researchers train on the new asset in days, with its rights enforced from the first run.
Closing the loop from data to model gain
Today. An ablation shows a gain, but tracing it back to a specific asset, filter version and mixture weight takes days of notebook archaeology. Acquisition ROI is a guess.
With Data Workers. Data Context Wizard links every asset to its dataset versions, every dataset version to the training runs that used it, and every run to its eval results. The acquisition dashboard shows cost per unique usable token and the measured eval delta per asset.
What changes. The next acquisition is prioritized on evidence: which domains, formats and partners actually moved the model.
Scaling 10x the data without 10x the cost
Today. Every new asset and experiment adds a full copy. Storage and processing bills outpace the corpus.
With Data Workers. Cost Savings & Data Cleanup deduplicates across purchased assets, retires stale intermediate copies once lineage proves nothing depends on them, tiers cold data and right-sizes processing compute.
What changes. The corpus grows by an order of magnitude while cost per usable token falls.
Agents working together: A seller's 40 TB asset, vetted before the term sheet
The seller shares a 2 TB sample of a 40 TB enterprise asset on Monday. By Monday evening the acquisition lead has a report that would have taken two weeks: 31% of the sample near-duplicates content already in the corpus, 4% overlaps a held-out eval set and must be excluded, PII appears in two fields that the seller will need to remove, and one clause in the draft license would block use in post-training. The negotiation starts from unique usable tokens, and the price reflects them.
Minute by minute
Illustrative example. A seller shares a 2 TB sample of a 40 TB asset.
| Time | Who | What happens |
|---|---|---|
| Mon 09:00 | Seller | Sample shared |
| 09:30 | Data Ingestion agent | Formats, volumes and languages profiled |
| 11:00 | Data Security agent | scan_pii: two fields with PII |
| 13:00 | Quality Monitoring agent | 31% corpus overlap, 4% eval contamination |
| 15:00 | Quality Monitoring agent | License clause blocking post-training flagged |
| 17:00 | Spellbook Data Catalog | Vetting report to the acquisition lead |
The agents on the job
| Agent | What it does for data acquisition and training data teams | Tools it calls |
|---|---|---|
| Connectors and Data Ingestion | Pull seller samples and full assets from data rooms and buckets, and profile every load | get_connector_status, trigger_airflow_dag, get_dbt_run_history |
| Data Security | Scans for PII and sensitive content before anything enters a corpus | scan_pii, review_pii_writes, pci_card_data_masking |
| Quality Monitoring | Measures quality signals, corpus overlap and eval contamination per asset | set_sla, run_quality_check, get_anomalies, create_quality_tests_for_pipeline |
| Schema Evolution | Normalizes hundreds of source formats into the training data standard | detect_schema_change, validate_schema_compatibility, generate_migration, rollback_migration |
| Pipeline Building | Runs versioned filter, dedup, decontamination and mixing pipelines with tests | generate_pipeline, validate_pipeline, sandbox_test, deploy_pipeline |
| Data Access & Governance | Enforces license scope and retention on every dataset and training run | provision_access, check_policy, request_governance_review, generate_audit_report |
| Cost Savings & Data Cleanup | Deduplicates across assets, tiers storage and retires stale copies | get_snowflake_usage, get_cost_enrichment, detect_dead_assets |
Data Context Wizard holds provenance, license terms and lineage from asset to eval delta. Autonomous Data-Conductor assembles vetting reports and verifies processing missions. Spellbook Data Catalog gives researchers a governed catalog of every asset and dataset version, with rights on every page.
What goes on autopilot
Once a domain has earned it, this work runs on its own, and each item removes a problem your team deals with today:
- •Seller samples are profiled for overlap, contamination, PII and license flags before negotiation, so you stop paying for data you already own.
- •Signed assets are processed into training-ready tokens as a mission, so researchers train on new data in days.
- •Cross-asset duplicates are removed and storage tiered, so cost per usable token falls as the corpus grows.
Where your people stay in charge
Agents never get blanket access. Each domain has its own autonomy level, set by you, and it only rises when the receipts show the agents are reliable. Everything starts at L1, observing only; reversible upkeep such as reruns, staging fixes, documentation and cleanup is usually first to run on its own.
- •Changes to released training mixes and license scope always goes to a named approver: these carry legal and model risk, so a named lead approves.
- •Deal terms and pricing: these stay with people.
Once a deployment has matured, here is how that ladder plays out across task classes:
Sample profiling closes autonomously 86% of the time. For Released mixes and license scope, autonomous closure is 0% by design: every change is proposed with a full diff and approved by a named person.
What happens when an agent is wrong
- •At L2 propose, nothing changes until a named person approves. The proposal shows the full diff, the blast radius and how to roll it back. Unanswered requests expire and escalate.
- •At L3 act, reversibly, every change keeps a rollback point, and Conductor verifies the downstream number after the change. If verification fails, the change is reversed and the domain drops a level.
- •No agent approves or promotes its own work. Autonomy only grows on evidence, per repair class, and repeated errors shrink scope automatically.
- •One org-wide stop halts all autonomous dispatch, and every action lands in a hash-chained audit trail you can export with generate_audit_report.
How it fits your stack
Data Workers connects to what you already run: Seller data rooms and buckets; object storage; Spark or Ray; dedup and filter pipelines; training run trackers; eval stores. Each agent is an MCP server, so the same agents work inside Claude Code, Cursor or Codex and in Slack or Teams through Spellbook. Nothing migrates.
- •Runs in your infrastructure with your credentials and your model key. The hosted Autonomous Data-Conductor sees workflow metadata only, never rows.
- •Uses the access you already govern. Agents act under scoped service identities and your existing roles; every grant goes through provision_access and is logged.
- •Proposes changes as diffs in your dbt project, orchestrator and warehouse, with blast radius, tests and a rollback point, and applies them only at the autonomy level you set for that domain.
- •Open-source core. The Apache-2.0 community edition ships 11 agents and 160+ MCP tools, so your engineers can read exactly what the agents call.
Governance and regulation
Acquired data carries license, privacy and provenance obligations. Data Workers records terms against the asset, enforces scope on every run and logs every use.
Data Workers doesn't make you compliant; your controls, people and assessors do. It supplies controls and evidence wherever agents touch data. Not legal advice.
The alternatives data acquisition and foundation model data teams weigh
Build it yourselves. A strong team can wire a coding agent to MCP servers for the warehouse and dbt. The work is everything around it: routing approvals to the right owner, blast radius, rollback, receipts, and current context about every other system.
Observability, catalogs and point tools. Observability alerts, catalogs document, and quality tools test. Data Workers takes their signals, diagnoses, proposes the fix, verifies it and leaves the receipt.
Platform-native agents. Snowflake made its managed MCP server and Cortex Agents generally available on Nov 4, 2025, and Databricks made Genie One generally available on Jun 16, 2026. They are excellent inside their platform; this work crosses the source systems, orchestrator, warehouse and BI, and Data Workers works across all of them.
In this space specifically. Data marketplaces and vendors sell assets; Data Workers vets them before purchase and turns them into governed tokens after.
The case for your CTO
What this replaces. Two weeks of manual profiling per seller sample, overlap discovered after purchase and month-long processing queues.
What your team gets back. An acquisition team that negotiates on unique usable tokens and gets new assets into training in days.
Integration effort. Connecting is configuration, not a migration. The agents attach over MCP to the warehouse, orchestrator, dbt project and source systems you already run, under the roles you already govern, inside your own infrastructure. Your engineers can read every tool the agents call in the open-source core.
Risk and control. Autonomy is set per domain. Everything starts at L1, observing only; reversible work moves up only when the receipts justify it; changes to released training mixes and license scope stay at L2 with a named approver; and one org-wide stop halts all autonomous work.
Why now. Model scaling now depends on purchased data.
The first win. Vet the next three seller samples. The pilot below lays out the scope, what you see each week and the cost.
The climb: from assisted to autonomous to agentic
Our thesis is that the data platform climbs the way analytics did, in stages, and that the hard part of data work was never knowing what to do but doing it. For data acquisition and foundation model data teams, the climb looks like this:
- •Assisted. Agents work alongside your team. They read the whole stack, show what needs doing and propose every fix with a receipt; changes to released training mixes and license scope always go to a named approver.
- •Autonomous, under your governance. Routine work runs on its own: seller samples are profiled for overlap, contamination, PII and license flags before negotiation, signed assets are processed into training-ready tokens as a mission, and cross-asset duplicates are removed and storage tiered. Your team governs the agents instead of doing the upkeep.
- •The agentic enterprise. An acquisition team that negotiates on unique usable tokens and gets new assets into training in days. People set direction, and the platform carries it out.
You choose the altitude, every step is visible and reversible, and the pilot below is the first rung.
The pilot: how data acquisition and foundation model data teams start
The pilot is how your team proves the value on its own data before committing.
- •What we connect. Seller data rooms and buckets; object storage; Spark or Ray; dedup and filter pipelines; training run trackers; eval stores, over MCP, inside your infrastructure. Nothing migrates.
- •Scope. Vet the next three seller samples.
- •First week. Agents run at L1 across the scoped domain. They map every pipeline, definition and lineage path and show the fixes they would have made, with receipts, so your team sees the value before anything changes.
- •By week two. Seller samples are profiled for overlap, contamination, PII and license flags before negotiation and signed assets are processed into training-ready tokens as a mission start running on reversible work. Changes to released training mixes and license scope stay at L2 with a named approver.
- •At day 30. We review the impact metrics against your baseline:
- •Asset traced to eval delta: from about 2 days to seconds
- •License change enforced: from about 2 weeks to ~1 h
- •Duplicate corpus copies retired: from about a month to hours
- •the share of sample profiling closed autonomously, targeting 86% as the deployment matures
- •What you decide. Whether to expand to the next domain. The pilot fee is credited in full against the first year.
- •Cost. Start with a pilot ($7,500 one-time, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.
The pilot is the start of a longer curve:
By month 12, more than 46% of agent actions run fully autonomously and manual work falls below 10%, while license-scope changes stay with a named approver.
FAQ
Can it vet before we sign? Yes, on a sample.
Does it enforce license scope? On every dataset and run.
Can it show ROI per asset? Lineage ties assets to eval deltas.
Where do the agents run? In your infrastructure, with your credentials and model key. The hosted Conductor sees workflow metadata only.
Do we have to migrate anything? No. Your warehouse, dbt project, orchestrator, catalog and BI stay where they are.
Sources
- •AfterQuery, Data Acquisition Lead: https://jobs.ashbyhq.com/AfterQuery/8041b75b-aeda-4363-be56-87ecabef682b
- •Data Workers pricing: https://dataworkers.io/pricing/
- •Data Workers open-source repository (tool registrations): https://github.com/DataWorkersProject/dataworkers-claw-community
- •Snowflake, managed MCP server and Cortex Agents GA release note (Nov 4, 2025): https://docs.snowflake.com/en/release-notes/2025/other/2025-11-04-cortex-agents-mcp
- •Databricks, Genie One launch (Jun 16, 2026): https://www.databricks.com/company/newsroom/press-releases/databricks-launches-genie-one-all-new-agentic-coworker-every-team