Best practices for agentic clinical data: a field guide for EHR and research data teams
Clinical data teams carry the highest stakes in data: patient privacy, research validity and care decisions. This report covers how they use agents to make EHR data usable in days instead of quarters.
Health systems sit on decades of EHR data, and research, quality and operations teams all want it. Research warehouses on Clarity, Caboodle and OMOP, enterprise data groups inside health IT, and real-world data companies aggregating across sites all face the same problem: making clinical data usable is slow, manual and privacy-constrained.
The pressure is rising. Health systems are joining research networks and Epic Cosmos, payers and life-sciences partners want linked data, and AI pilots need governed clinical concepts. Each of those multiplies the onboarding, mapping and access work.
The repeatable parts of that work, connecting, profiling, classifying PHI, mapping codes, writing quality rules and documenting lineage, are what agents should own. Clinicians and informaticists should spend their time on the judgment calls only they can make.
The old world
A new partner hospital joined the research network three months ago and its data is still not usable. The ADT feed is live, but lab codes are local, the PHI review is pending, and the engineer who knows HL7 is on another project. A cardiology researcher's cohort request sits behind 30 others for three weeks.
The new world
With Data Workers, onboarding ran as one mission: connected, profiled, PHI classified before anyone opened a row, policy applied, mapped to OMOP with tests, documented with lineage. One clinician reviewed the 3% of lab codes mapped with low confidence. The cohort request runs on governed concepts and arrives in two days.
Here is that new world as a chain of hand-offs between agents:
The full playbook walks through this scenario and three more use cases in depth.
The rest of this report walks through the 6 practices that make that morning possible. At a glance:
The practices, one by one
1. Classify PHI before you model
Privacy review at the end of a project costs months.
The old world. Classification happens per project, late.
The new world. Data Security classifies every field on landing and scrubs sensitive values before anything reaches context.
In practice. A new feed's PHI is tagged before the first model is built.
What to measure. Tables used before PHI classification.
2. Standardize early
Comparable data across sites needs a common model.
The old world. Each site mapped by hand.
The new world. Pipeline Building maps each source into OMOP or your canonical model with tests and confidence scores.
In practice. Only low-confidence mappings go to an informaticist.
What to measure. Share of codes mapped automatically.
Practices like these compound. Across the routine work they cover, here is the difference in elapsed time:
The biggest change is PHI classification on a new feed: from about a week to minutes. Work that waited in queues finishes in minutes or hours, because agents detect, fix and verify in one pass.
3. Track unmapped codes as a backlog
Local codes quietly reduce cohort accuracy.
The old world. Nobody knows how many remain.
The new world. A weekly routine surfaces unmapped codes by volume.
In practice. The top 50 unmapped lab codes are fixed in a week.
What to measure. Encounters affected by unmapped codes.
4. Govern clinical concepts
A readmission should mean the same thing in every study.
The old world. Each analyst re-interprets.
The new world. Context Wizard holds governed concept definitions with provenance.
In practice. Two studies use the same diabetic cohort definition.
What to measure. Concepts with a governed definition.
5. Automate access by policy
Researchers should not wait weeks for approved access.
The old world. Tickets and manual checks.
The new world. Data Access & Governance applies IRB and role policy and logs every grant.
In practice. De-identified access granted the day IRB approval lands.
What to measure. Days to policy-compliant access.
That balance, autonomy where work is reversible and a named approver where it matters, shows up clearly by task class:
Data dictionary upkeep closes autonomously 86% of the time. For Identified-data access, autonomous closure is 0% by design: every change is proposed with a full diff and approved by a named person.
6. Onboard sites as a mission
Every new site follows the same steps.
The old world. A quarter per site.
The new world. Autopilot runs connect, land, classify, govern, model, test and document as one mission.
In practice. A new site is live in about a week.
What to measure. Days to onboard a site.
Designing the autonomy ladder
Autonomy is set per task class, not per tool. This is where we would set each class for a mature deployment, and why:
| Task class | Closed autonomously | Escalated | Handled by a person | Where we set it | Why |
|---|---|---|---|---|---|
| Data dictionary upkeep | 86% | 10% | 4% | L4 once the receipts are clean | The work is reversible and repeatable |
| Nightly load verification | 78% | 15% | 7% | L3: acts and keeps a rollback point | Most cases are routine; the rest escalate with evidence |
| PHI tagging | 64% | 28% | 8% | L3: acts and keeps a rollback point | Most cases are routine; the rest escalate with evidence |
| Quality rules | 58% | 30% | 12% | L3: acts and keeps a rollback point | Most cases are routine; the rest escalate with evidence |
| Code mapping | 40% | 44% | 16% | L3 for routine cases, L2 for the rest | Judgment is needed often enough to keep a person close |
| Identified-data access | 0% | 85% | 15% | L2: always proposed to a named approver | The number carries accountability outside the data team |
The operating model: who owns what
Agents change who does the work, not who is accountable for it:
| Role | Owns | What the agents take off their plate |
|---|---|---|
| Clinical Data Product Manager | Standards, autonomy levels per domain and which domain goes next | Assembling status, chasing owners, keeping the catalog current |
| Domain owners and approvers | New code mappings and access to identified data | Preparing the diff, blast radius and rollback plan for every proposal |
| Data engineers | Reviewing agent proposals and building new data products | Every new field is classified for PHI the moment it lands; nightly loads are verified and reruns handled automatically |
The swarm behind the practices
No single agent delivers these practices. Each one is a hand-off between specialists, coordinated under the autonomy you set per domain.
| Agent | What it does for clinical data teams |
|---|---|
| Connectors and Data Ingestion | Land ADT, HL7, FHIR and Clarity extracts, and profile every load |
| Data Security | Classifies PHI field by field and scrubs it before it reaches context |
| Data Access & Governance | Applies IRB and role policies to every table and logs every grant |
| Pipeline Building | Maps sources into OMOP or your canonical model and writes tests |
| Quality Monitoring | Encounter volume, lab range, ordering and duplicate-patient checks |
| Data Context & Catalog | Keeps the data dictionary and lineage current, from message to cohort |
Context Wizard holds governed concept definitions, such as a diabetic patient or a readmission, so cohorts mean the same thing in every study. Conductor keeps nightly loads fresh and verified. Spellbook lets researchers find datasets and request access in plain English.
What disappears
The clearest sign the practices are working is what stops happening:
- •Months-long site onboarding.
- •PHI discovered after modeling.
- •Three-week cohort queues.
- •Out-of-date data dictionaries.
- •Inconsistent cohort definitions.
- •Access waits after IRB approval.
The magic is not any single agent. It is that the back office simply keeps running. Bad data is caught where it enters, not where it is read. Every new table gets quality rules and documentation without anyone writing them. Access follows policy instead of a queue. Fixes are rerun, verified and explained before anyone opens a dashboard. Cost is cleaned up every week instead of every budget cycle. And every one of those actions leaves a signed receipt, so the team can see exactly what happened and roll anything back. For the people on the team, the experience is quieter mornings, fewer tickets and more time for the products the business asked for.
A product manager's scorecard
If you own this data as a product, these are the numbers to bring to every quarterly review:
- •Days to onboard a site
- •Cohort request cycle time
- •Share of codes mapped automatically
- •Tables used before PHI classification (target zero)
- •Days to policy-compliant access
- •Share of upkeep autonomous
The climb: from assisted to autonomous to agentic
Our thesis is that the data platform climbs the way analytics did, in stages, and that the hard part of data work was never knowing what to do but doing it. For clinical and EHR data teams, the climb looks like this:
- •Assisted. Agents work alongside your team. They read the whole stack, show what needs doing and propose every fix with a receipt; new code mappings and access to identified data always go to a named approver.
- •Autonomous, under your governance. Routine work runs on its own: every new field is classified for PHI the moment it lands, nightly loads are verified and reruns handled automatically, and the data dictionary and lineage update as the agents work. Your team governs the agents instead of doing the upkeep.
- •The agentic enterprise. Clinical informaticists spending their time on the mappings that need clinical judgment, and researchers getting cohorts in days. People set direction, and the platform carries it out.
You choose the altitude, every step is visible and reversible, and the pilot below is the first rung.
The pilot: how clinical and EHR data teams start
The pilot is how your team proves the value on its own data before committing.
- •What we connect. Epic Clarity and Caboodle; HL7 and FHIR feeds via an integration engine; Snowflake, Databricks or a research warehouse; OMOP; dbt, over MCP, inside your infrastructure. Nothing migrates.
- •Scope. One partner site onboarded as a mission.
- •First week. Agents run at L1 across the scoped domain. They map every pipeline, definition and lineage path and show the fixes they would have made, with receipts, so your team sees the value before anything changes.
- •By week two. Every new field is classified for PHI the moment it lands and nightly loads are verified and reruns handled automatically start running on reversible work. New code mappings and access to identified data stay at L2 with a named approver.
- •At day 30. We review the impact metrics against your baseline:
- •PHI classification on a new feed: from about a week to minutes
- •Finding the right dataset: from about 8 hours to 30 s
- •Local lab code mapped: from about 2 days to under 1 h
- •the share of data dictionary upkeep closed autonomously, targeting 86% as the deployment matures
- •What you decide. Whether to expand to the next domain. The pilot fee is credited in full against the first year.
- •Cost. Start with a pilot ($7,500 one-time, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.
The pilot is the start of a longer curve:
By month 12, more than 34% of agent actions run fully autonomously and manual work falls below 10%, while identified-data access stay with a named approver.
Read the full playbook: Clinical, EHR and research data teams