Clinical and R&D data teams: stop rebuilding study datasets, start reusing every trial you have run

Key takeaways

• •Who this is for. Teams that own trial data (SDTM, ADaM), preclinical and discovery data and the platforms that pool them for reuse.
• •The problem today. Six-week queues for every locked study, study-by-study privacy reviews and incomplete metadata.
• •What changes. Studies are profiled, mapped and pooled at database lock, so science never waits on a backlog. Anonymization rules apply consistently, so only true exceptions need a privacy expert.
• •What stays with your people. Anonymization exceptions and identified-data access: these protect patients, so the governance lead approves.
• •The result. An R&D data platform where every trial becomes reusable the week it closes.

The PRD for agentic clinical and R&D data teams

This is the requirements document we would write with you for your team. Everything below expands on it.

What going fully agentic and autonomous means for clinical and R&D data teams

• •Put your data pipeline on autopilot. Studies are profiled, mapped and pooled at database lock, so science never waits on a backlog.
• •Run the data back office without the busywork. The work that fills the week today moves to agents: six-week queues for every locked study, study-by-study privacy reviews and incomplete metadata.
• •Scale data without scaling headcount. Anonymization rules apply consistently and consent basis and lineage are documented for every pooled record, so the data you manage grows without the team having to.
• •Get the plumbing off your plate. An R&D data platform where every trial becomes reusable the week it closes.

Who this is for

Teams that own trial data (SDTM, ADaM), preclinical and discovery data and the platforms that pool them for reuse. They sit in enterprise R&D data platforms, clinical data management or a data science and AI organization. Titles include Head of Data Engineering & Data Products, Clinical Data SME, Data Product Owner (R&D) and Head of Data Governance, Anonymization & Quality.

The shift

Pharma companies have run thousands of trials, and most of that data sits in study-specific silos. Pooling it for AI and secondary research needs standardization, anonymization and governance at a scale no team can staff by hand. Agents make pooling continuous: every new study is profiled, standardized, anonymized and documented as it closes.

The product you ship

• •Pooled SDTM and ADaM datasets across studies
• •Anonymized data for secondary use
• •Discovery and preclinical data products
• •Governed metadata and lineage for regulators and scientists

Where the week goes

• •Each study uses slightly different standards.
• •Anonymization is reviewed study by study.
• •Scientists wait weeks for pooled datasets.
• •Metadata is incomplete.
• •Access requires manual approvals per study.

Add it up and upkeep is most of the team's week. Here is how a typical week splits today, and once agents own the back office:

Time spent on new data products moves from 14% of the week to 50%. Most of the rest becomes review: people steer and approve agent work instead of doing it by hand.

Then and now

These are the impact metrics we move for clinical and R&D data teams:

• •Access decision: from about a week to ~1 h
• •Metadata for a study: from about 3 days to under 1 h
• •Anonymization review: from about a week to hours

Across the routine work in this domain:

The biggest change is Access decision: from about a week to ~1 h.

Use cases, in depth

Pooling 2,000 studies

Today. Each study is mapped by hand to a common model.

With Data Workers. Data Ingestion profiles each study, Schema Evolution maps deviations from SDTM/ADaM, Pipeline Building pools with tests, Data Context & Catalog documents lineage.

What changes. Studies join the pool as they close.

Anonymization as code

Today. Every secondary-use request needs a privacy review.

With Data Workers. Data Security applies anonymization rules consistently; exceptions go to the governance lead.

What changes. Secondary use in days.

A scientist's pooled question

Today. A data scientist needs adverse events across 40 studies.

With Data Workers. Context Wizard resolves governed definitions; Access & Governance checks consent and purpose; the pooled extract is built and tested.

What changes. Science moves at the speed of the question.

Agents working together: A closed study joins the pool

Minute by minute

This is an illustrative example. Locked studies land in a Databricks R&D lakehouse; the pooled model serves data science.

Without the agents, the study waits weeks in a queue.

The agents on the job

Data Context Wizard keeps one governed graph of definitions, lineage and owners that every agent and person answers from. Autonomous Data-Conductor runs each job from detection to a verified result. Spellbook Data Catalog is where your team approves, audits and rolls back, and asks questions in plain English.

What goes on autopilot

Once a domain has earned it, this work runs on its own, and each item removes a problem your team deals with today:

• •Studies are profiled, mapped and pooled at database lock, so science never waits on a backlog.
• •Anonymization rules apply consistently, so only true exceptions need a privacy expert.
• •Consent basis and lineage are documented for every pooled record, so reuse is defensible.

Where your people stay in charge

Agents never get blanket access. Each domain has its own autonomy level, set by you, and it only rises when the receipts show the agents are reliable. Everything starts at L1, observing only; reversible upkeep such as reruns, staging fixes, documentation and cleanup is usually first to run on its own.

• •Anonymization exceptions and identified-data access always goes to a named approver: these protect patients, so the governance lead approves.
• •Scientific and regulatory decisions: these stay with people.

Once a deployment has matured, here is how that ladder plays out across task classes:

Metadata upkeep closes autonomously 88% of the time. For Identified-data access, autonomous closure is 0% by design.

What happens when an agent is wrong

• •At L2 propose, nothing changes until a named person approves. The proposal shows the full diff, the blast radius and how to roll it back. Unanswered requests expire and escalate.
• •At L3 act, reversibly, every change keeps a rollback point, and Conductor verifies the downstream number after the change. If verification fails, the change is reversed and the domain drops a level.
• •No agent approves or promotes its own work. Autonomy only grows on evidence, per repair class, and repeated errors shrink scope automatically.
• •One org-wide stop halts all autonomous dispatch, and every action lands in a hash-chained audit trail you can export with generate_audit_report.

How it fits your stack

Data Workers connects to what you already run: Databricks or Snowflake; EDC and CTMS exports; SAS and R programming environments; metadata repositories; dbt or Spark pipelines. Each agent is an MCP server, so the same agents work inside Claude Code, Cursor or Codex and in Slack or Teams through Spellbook. Nothing migrates.

• •Runs in your infrastructure with your credentials and your model key. The hosted Autonomous Data-Conductor sees workflow metadata only, never rows.
• •Uses the access you already govern. Agents act under scoped service identities and your existing roles; every grant goes through provision_access and is logged.
• •Proposes changes as diffs in your dbt project, orchestrator and warehouse, with blast radius, tests and a rollback point, and applies them only at the autonomy level you set for that domain.
• •Open-source core. The Apache-2.0 community edition ships 11 agents and 160+ MCP tools, so your engineers can read exactly what the agents call.

Governance and regulation

Trial data reuse depends on consent, anonymization and traceability for regulators. Data Workers applies anonymization rules consistently, documents consent basis and lineage, and logs every access; your privacy and regulatory teams decide sufficiency.

Data Workers doesn't make you compliant; your controls, people and assessors do. It supplies controls and evidence wherever agents touch data. Not legal advice.

The alternatives clinical and R&D data teams weigh

Build it yourselves. A strong team can wire a coding agent to MCP servers for the warehouse and dbt. The work is everything around it: routing approvals to the right owner, blast radius, rollback, receipts, and current context about every other system.

Observability, catalogs and point tools. Observability alerts, catalogs document, and quality tools test. Data Workers takes their signals, diagnoses, proposes the fix, verifies it and leaves the receipt.

Platform-native agents. Snowflake made its managed MCP server and Cortex Agents generally available on Nov 4, 2025, and Databricks made Genie One generally available on Jun 16, 2026. They are excellent inside their platform; this work crosses the source systems, orchestrator, warehouse and BI, and Data Workers works across all of them.

In this space specifically. Clinical data repositories store studies; Data Workers standardizes, anonymizes and pools them continuously and keeps the metadata current.

The case for your CTO

What this replaces. Six-week queues for every locked study, study-by-study privacy reviews and incomplete metadata.

What your team gets back. An R&D data platform where every trial becomes reusable the week it closes.

Integration effort. Connecting is configuration, not a migration. The agents attach over MCP to the warehouse, orchestrator, dbt project and source systems you already run, under the roles you already govern, inside your own infrastructure. Your engineers can read every tool the agents call in the open-source core.

Risk and control. Autonomy is set per domain. Everything starts at L1, observing only; reversible work moves up only when the receipts justify it; anonymization exceptions and identified-data access stay at L2 with a named approver; and one org-wide stop halts all autonomous work.

Why now. AI programs need pooled data now, and manual pooling cannot keep up.

The first win. Pool 50 recent studies at L2 for anonymization exceptions. The pilot below lays out the scope, what you see each week and the cost.

The climb: from assisted to autonomous to agentic

Our thesis is that the data platform climbs the way analytics did, in stages, and that the hard part of data work was never knowing what to do but doing it. For clinical and R&D data teams, the climb looks like this:

• •Assisted. Agents work alongside your team. They read the whole stack, show what needs doing and propose every fix with a receipt; anonymization exceptions and identified-data access always go to a named approver.
• •Autonomous, under your governance. Routine work runs on its own: studies are profiled, mapped and pooled at database lock, anonymization rules apply consistently, and consent basis and lineage are documented for every pooled record. Your team governs the agents instead of doing the upkeep.
• •The agentic enterprise. R&D data platforms will pool every study at lock, anonymize by rule and let scientists ask questions across decades of trials in days. The pharma companies that get there will run AI on all of their evidence, not the latest study.

You choose the altitude, every step is visible and reversible, and the pilot below is the first rung.

The pilot: how clinical and R&D data teams start

The pilot is how your team proves the value on its own data before committing.

• •What we connect. Databricks or Snowflake; EDC and CTMS exports; SAS and R programming environments; metadata repositories; dbt or Spark pipelines, over MCP, inside your infrastructure. Nothing migrates.
• •Scope. Pool 50 recent studies at L2 for anonymization exceptions.
• •First week. Agents run at L1 across the scoped domain. They map every pipeline, definition and lineage path and show the fixes they would have made, with receipts, so your team sees the value before anything changes.
• •By week two. Studies are profiled, mapped and pooled at database lock and anonymization rules apply consistently start running on reversible work. Anonymization exceptions and identified-data access stay at L2 with a named approver.
• •At day 30. We review the impact metrics against your baseline:

• •Access decision: from about a week to ~1 h
• •Metadata for a study: from about 3 days to under 1 h
• •Anonymization review: from about a week to hours
• •the share of metadata upkeep closed autonomously, targeting 88% as the deployment matures

• •What you decide. Whether to expand to the next domain. The pilot fee is credited in full against the first year.
• •Cost. Start with a pilot ($7,500 one-time, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.

The pilot is the start of a longer curve:

By month 12, more than 34% of agent actions run fully autonomously while identified-data access stay with a named approver.

FAQ

Does it support CDISC standards? It maps to your SDTM/ADaM conventions and flags deviations for review.

Who approves anonymization exceptions? A named governance lead, always.

Does data leave our environment? No; agents run in your infrastructure.

Where do the agents run? In your infrastructure, with your credentials and model key. The hosted Conductor sees workflow metadata only.

Do we have to migrate anything? No. Your warehouse, dbt project, orchestrator, catalog and BI stay where they are.

Sources

• •Novartis, data42 program: https://www.novartis.com/stories/data42-program-shows-novartis-intent-go-big-data-and-digital.md
• •Novartis, Head of data42 Data Management & Data Products posting: https://builtindublin.ie/job/head-data42-data-management-data-products/7332662
• •Data Workers pricing: https://dataworkers.io/pricing/
• •Data Workers open-source repository (tool registrations): https://github.com/DataWorkersProject/dataworkers-claw-community
• •Snowflake, managed MCP server and Cortex Agents GA release note (Nov 4, 2025): https://docs.snowflake.com/en/release-notes/2025/other/2025-11-04-cortex-agents-mcp
• •Databricks, Genie One launch (Jun 16, 2026): https://www.databricks.com/company/newsroom/press-releases/databricks-launches-genie-one-all-new-agentic-coworker-every-team

Related: Best practices for agentic clinical trial and R&D data