Best practices for agentic data acquisition: a field guide for foundation model data teams
Scaling foundation models now means acquiring ten times more data and proving each asset made the model better. This report covers how acquisition and training data teams turn that into one governed pipeline run by agents.
Data acquisition has become a core function of every lab that trains foundation models. With the open web largely mined, the frontier is proprietary enterprise data, licensed content and expert data, bought deal by deal, often in seven figures. Labs and the applied data labs that serve them are hiring their first Data Acquisition Leads to run it.
The value of each deal depends on questions that are hard to answer quickly: how much of this asset is new, how clean it is, what we may do with it, and whether it improved the model. Today those answers arrive after the money is spent.
The teams that scale fastest move those answers to the front: vet before signing, process in days, enforce rights on every run, and measure every asset against the eval deltas it produced.
The old world
A seller shares a 2 TB sample of a 40 TB asset. Two weeks of manual profiling later, the team signs. After purchase, a third of the asset turns out to duplicate content already in the corpus, part of it overlaps a held-out eval set, and a clause in the license blocks post-training use. Processing takes another month.
The new world
With Data Workers, the sample was vetted in hours: unique usable tokens, quality, PII exposure, corpus overlap, eval contamination and license flags. The price reflected what was actually new. After signing, processing ran as a mission and researchers trained on the asset within days, with rights enforced from the first run.
Here is that new world as a chain of hand-offs between agents:
The full playbook walks through this scenario and three more use cases in depth.
The rest of this report walks through the 6 practices that make that morning possible. At a glance:
The practices, one by one
1. Vet before you sign, on unique usable tokens
Raw terabytes are the wrong unit.
The old world. Manual profiling, overlap found after purchase.
The new world. A vetting mission profiles the sample, measures overlap and contamination, scans PII and flags license terms.
In practice. A 40 TB asset is priced on its 31% of genuinely new tokens.
What to measure. Hours from sample to vetting report.
2. Record rights in the data, not just the contract
Usage scope must follow the data into every mix.
The old world. Contracts and spreadsheets.
The new world. Access & Governance enforces license scope on every dataset and run.
In practice. A post-training restriction is enforced automatically.
What to measure. Assets with rights enforced.
Practices like these compound. Across the routine work they cover, here is the difference in elapsed time:
The biggest change is License change enforced: from about 2 weeks to ~1 h. Work that waited in queues finishes in minutes or hours, because agents detect, fix and verify in one pass.
3. Decontaminate on every run
New assets are the main source of eval leakage.
The old world. Found when scores look too good.
The new world. Conductor runs a contamination check before every training run.
In practice. A 4% eval overlap is excluded before training.
What to measure. Runs with contamination checks.
4. Version filters and mixtures
You should know what every model saw.
The old world. Notebook archaeology.
The new world. Pipeline Building versions every filter and mix; Context Wizard links them to runs.
In practice. An eval delta is traced to one asset and one filter version.
What to measure. Runs with full lineage.
5. Tie every asset to a model gain
Acquisition budgets should follow measured impact.
The old world. ROI estimated, if at all.
The new world. Lineage from asset to dataset to run to eval delta, reported per asset and partner.
In practice. The next budget shifts toward the domains that moved the model.
What to measure. Assets with measured eval impact.
That balance, autonomy where work is reversible and a named approver where it matters, shows up clearly by task class:
Sample profiling closes autonomously 86% of the time. For Released mixes and license scope, autonomous closure is 0% by design: every change is proposed with a full diff and approved by a named person.
6. Scale data, not cost
10x the data should not mean 10x the bill.
The old world. Every experiment copies the corpus.
The new world. Cost Savings & Data Cleanup deduplicates across assets and tiers storage.
In practice. Cost per usable token falls as the corpus grows.
What to measure. Cost per unique usable token.
Designing the autonomy ladder
Autonomy is set per task class, not per tool. This is where we would set each class for a mature deployment, and why:
| Task class | Closed autonomously | Escalated | Handled by a person | Where we set it | Why |
|---|---|---|---|---|---|
| Sample profiling | 86% | 10% | 4% | L4 once the receipts are clean | The work is reversible and repeatable |
| Contamination checks | 84% | 12% | 4% | L4 once the receipts are clean | The work is reversible and repeatable |
| Format normalization | 70% | 22% | 8% | L3: acts and keeps a rollback point | Most cases are routine; the rest escalate with evidence |
| Dedup and filter reruns | 72% | 20% | 8% | L3: acts and keeps a rollback point | Most cases are routine; the rest escalate with evidence |
| PII scanning | 66% | 26% | 8% | L3: acts and keeps a rollback point | Most cases are routine; the rest escalate with evidence |
| Released mixes and license scope | 0% | 88% | 12% | L2: always proposed to a named approver | The number carries accountability outside the data team |
The operating model: who owns what
Agents change who does the work, not who is accountable for it:
| Role | Owns | What the agents take off their plate |
|---|---|---|
| Data Acquisition Lead | Standards, autonomy levels per domain and which domain goes next | Assembling status, chasing owners, keeping the catalog current |
| Domain owners and approvers | Changes to released training mixes and license scope | Preparing the diff, blast radius and rollback plan for every proposal |
| Data engineers | Reviewing agent proposals and building new data products | Seller samples are profiled for overlap, contamination, PII and license flags before negotiation; signed assets are processed into training-ready tokens as a mission |
The swarm behind the practices
No single agent delivers these practices. Each one is a hand-off between specialists, coordinated under the autonomy you set per domain.
| Agent | What it does for data acquisition and training data teams |
|---|---|
| Connectors and Data Ingestion | Pull seller samples and full assets from data rooms and buckets, and profile every load |
| Data Security | Scans for PII and sensitive content before anything enters a corpus |
| Quality Monitoring | Measures quality signals, corpus overlap and eval contamination per asset |
| Schema Evolution | Normalizes hundreds of source formats into the training data standard |
| Pipeline Building | Runs versioned filter, dedup, decontamination and mixing pipelines with tests |
| Data Access & Governance | Enforces license scope and retention on every dataset and training run |
| Cost Savings & Data Cleanup | Deduplicates across assets, tiers storage and retires stale copies |
Data Context Wizard holds provenance, license terms and lineage from asset to eval delta. Autonomous Data-Conductor assembles vetting reports and verifies processing missions. Spellbook Data Catalog gives researchers a governed catalog of every asset and dataset version, with rights on every page.
What disappears
The clearest sign the practices are working is what stops happening:
- •Buying data the lab already owns.
- •Eval leakage from new assets.
- •License terms living only in contracts.
- •Month-long processing queues.
- •Guesswork about acquisition ROI.
- •Corpus copies per experiment.
The magic is not any single agent. It is that the back office simply keeps running. Bad data is caught where it enters, not where it is read. Every new table gets quality rules and documentation without anyone writing them. Access follows policy instead of a queue. Fixes are rerun, verified and explained before anyone opens a dashboard. Cost is cleaned up every week instead of every budget cycle. And every one of those actions leaves a signed receipt, so the team can see exactly what happened and roll anything back. For the people on the team, the experience is quieter mornings, fewer tickets and more time for the products the business asked for.
A product manager's scorecard
If you own this data as a product, these are the numbers to bring to every quarterly review:
- •Hours from sample to vetting report
- •Days from signature to training-ready tokens
- •Share of purchased tokens that are unique
- •Runs with contamination checks
- •Assets with measured eval impact
- •Cost per unique usable token
The climb: from assisted to autonomous to agentic
Our thesis is that the data platform climbs the way analytics did, in stages, and that the hard part of data work was never knowing what to do but doing it. For data acquisition and foundation model data teams, the climb looks like this:
- •Assisted. Agents work alongside your team. They read the whole stack, show what needs doing and propose every fix with a receipt; changes to released training mixes and license scope always go to a named approver.
- •Autonomous, under your governance. Routine work runs on its own: seller samples are profiled for overlap, contamination, PII and license flags before negotiation, signed assets are processed into training-ready tokens as a mission, and cross-asset duplicates are removed and storage tiered. Your team governs the agents instead of doing the upkeep.
- •The agentic enterprise. An acquisition team that negotiates on unique usable tokens and gets new assets into training in days. People set direction, and the platform carries it out.
You choose the altitude, every step is visible and reversible, and the pilot below is the first rung.
The pilot: how data acquisition and foundation model data teams start
The pilot is how your team proves the value on its own data before committing.
- •What we connect. Seller data rooms and buckets; object storage; Spark or Ray; dedup and filter pipelines; training run trackers; eval stores, over MCP, inside your infrastructure. Nothing migrates.
- •Scope. Vet the next three seller samples.
- •First week. Agents run at L1 across the scoped domain. They map every pipeline, definition and lineage path and show the fixes they would have made, with receipts, so your team sees the value before anything changes.
- •By week two. Seller samples are profiled for overlap, contamination, PII and license flags before negotiation and signed assets are processed into training-ready tokens as a mission start running on reversible work. Changes to released training mixes and license scope stay at L2 with a named approver.
- •At day 30. We review the impact metrics against your baseline:
- •Asset traced to eval delta: from about 2 days to seconds
- •License change enforced: from about 2 weeks to ~1 h
- •Duplicate corpus copies retired: from about a month to hours
- •the share of sample profiling closed autonomously, targeting 86% as the deployment matures
- •What you decide. Whether to expand to the next domain. The pilot fee is credited in full against the first year.
- •Cost. Start with a pilot ($7,500 one-time, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.
The pilot is the start of a longer curve:
By month 12, more than 46% of agent actions run fully autonomously and manual work falls below 10%, while license-scope changes stay with a named approver.
Read the full playbook: Data acquisition leads, training data engineering and pretraining data teams