Agentic genomics and multi-omics data: a guide for bioinformatics data teams

Key takeaways

• •Who this is for. The team that owns the metadata and tabular layer around sequencing, imaging and proteomics or transcriptomics data at a biopharma, research hospital, diagnostics lab or biobank.
• •The problem today. Sample metadata kept in spreadsheets, ontology terms that drift between cohorts, pipeline and annotation upgrades that change results without a visible break, and a hand-assembled paper trail every time a cohort goes into a paper, a submission or a partner release.
• •What changes. Agents map incoming sample sheets and assay tables into the warehouse, keep terms and definitions consistent, compare results across pipeline versions, and document every cohort with its inputs, lineage and consent basis.
• •What stays with your people. Agents do not call variants or analyse images, and they never decide consent scope, data-use agreements or external cohort release: your data access committee does, with the evidence already assembled.
• •The result. A genomics data team that ships reusable cohorts and clean results tables instead of reconciling spreadsheets and rebuilding last year's analysis from memory.

The PRD for agentic genomics, imaging and multi-omics data teams

This is the requirements document we would write with you for your team. Everything below expands on it.

What going fully agentic and autonomous means for genomics, imaging and multi-omics data teams

• •Put your data pipeline on autopilot. New sample sheets, assay exports and extracted imaging metadata are mapped into the warehouse the day they arrive, so a collaborator's new column does not stall the cohort build.
• •Run the data back office without the busywork. The work that fills the week today moves to agents: reconciling sample spreadsheets, remapping ontology terms, chasing which pipeline version produced which table and writing the same cohort methods paragraph again.
• •Scale data without scaling headcount. Every new assay, site and biobank partner adds tables, terms and access rules. Agents carry that growth in the metadata layer, so the team does not have to grow with the sample count.
• •Get the plumbing off your plate. A genomics data team that ships reusable cohorts and clean results tables instead of reconciling spreadsheets and rebuilding last year's analysis from memory.

Who this is for

The team that sits between the instruments and the scientists. At a biopharma it is the translational or R&D data platform group that makes sequencing, proteomics and imaging results queryable next to clinical and trial data. At a research hospital it is the imaging informatics and genomics data group that links specimens and scans to the clinical record. At a diagnostics lab it is the data engineering team behind assay results and reporting. At a biobank it is the data management team that tracks specimens, consent and releases to approved researchers.

The titles: Head of Genomics Data, Director of Bioinformatics Data Platforms, Imaging Informatics Lead, Biobank Data Manager, Principal Data Engineer (Translational), and the product owners who report to them. The people who run the sequencers and the analysis pipelines are your neighbours, not your replacement: this page is about the layer you own around their output. If your world also includes trial datasets, see the clinical and R&D data playbook; for the record-level clinical side, the clinical EHR data playbook.

The product you ship

• •Sample and specimen metadata: one governed record per sample, specimen or image study: source, collection, processing, storage, assay and status, in place of one spreadsheet per collaborator.
• •Assay results tables: variant tables, expression matrices, protein abundance tables and imaging-derived measurements, in the warehouse or lakehouse, with the pipeline and reference versions that produced them.
• •Cohorts: saved, versioned definitions that scientists reuse for analyses, papers and regulatory submissions.
• •Linkage to clinical data: keys and mappings that join specimens, scans and results to clinical tables, with the linkage rules documented.
• •Consent and data-use records: per-sample consent scope and the data-use terms attached to each dataset, held where access policy can read them.
• •Ontologies and controlled vocabularies: the mapped terms for phenotype, anatomy, tissue, disease and assay that make cohorts comparable across studies.
• •Release packages: the approved cohort, its data dictionary and its provenance, prepared for an internal group or an external partner.

What the agents touch, and what they leave alone

Be exact about scope, because this is where life sciences buyers are right to be sceptical.

• •Agents work on: sample and specimen metadata, assay and results tables in the warehouse or lakehouse, extracted DICOM header and slide metadata tables, ontology and mapping tables, pipeline run history and version tags, lineage, data quality, catalog documentation and access rules.
• •Agents do not do: alignment, variant calling, expression quantification, image segmentation or model-based image analysis. Those run in your workflow managers and analysis platforms, as they do today.
• •Agents do not read: raw reads, BAM or VCF files in object storage, or pixel data. They see what your pipelines write into tables, and the run records beside them.
• •Agents do not decide: consent scope, whether a data-use agreement permits a purpose, or whether a cohort leaves the building. Those go to a named owner.

Where the week goes

• •Sample metadata lives in spreadsheets. Each site, core and collaborator sends a sheet in its own layout, and someone maps it by hand.
• •Ontology drift. The same tissue or phenotype arrives under three spellings, two ontology releases and a free-text note, and cohorts quietly stop matching.
• •Pipeline and annotation upgrades change results silently. A new reference or annotation release reclassifies variants or shifts expression values, and nothing fails: the numbers just differ.
• •Consent differs per sample. One participant consented to cancer research only, another to any research, a third withdrew. Every cohort needs that filter applied and shown.
• •Cohorts must be reproducible. A reviewer or a regulator asks for the exact set behind a figure two years later.
• •Data-use agreements limit what each dataset may do. Terms live in PDFs, not next to the tables.
• •Imaging adds its own mess. DICOM headers are inconsistent across vendors and sites, and de-identification of headers and burned-in text is easy to miss.

Add it up and upkeep is most of the team's week. Here is how a typical week splits today, and once agents own the metadata back office:

Time spent on new data products moves from 14% of the week to 46%. Most of the rest becomes review: people steer and approve agent work instead of doing it by hand.

Then and now

These are the impact metrics we move for genomics, imaging and multi-omics data teams:

• •Cohort rebuilt as published: from about 40 hours to 2 hours
• •Result shift traced to a pipeline change: from about 24 hours to 30 minutes
• •Consent evidence for a release assembled: from about 12 hours to 1 hour

Across the routine work in this domain:

The biggest change is rebuilding a cohort exactly as published: from about 40 hours to 2 hours. Work that waited in queues finishes in minutes or hours, because agents detect, document and verify in one pass.

Use cases, in depth

A pipeline upgrade that moved the results

Today. The core upgrades an annotation release, re-runs a batch, and two weeks later a scientist notices that carrier counts in a gene panel no longer match last quarter's slide.

With Data Workers. Connectors read the run history and version tags, Quality Monitoring compares the new results tables with the prior release, Data Context & Catalog lists every cohort and analysis that depends on them, and Incident Debugging attributes the shift to the release change rather than to a data problem.

What changes. The bioinformatics lead sees the diff and the affected work on Monday morning, and decides which analyses stay on the old release.

A cohort you can rebuild exactly as published

Today. A figure is accepted, and 18 months later a reviewer, a regulator or a partner asks for the exact participants and the exact data versions behind it.

With Data Workers. Data Context & Catalog traces every source behind the cohort, Pipeline Building expresses the cohort as a tested, versioned model, Quality Monitoring adds checks for duplicates and missing consent flags, and Data Access & Governance checks the cohort against the access policy and prepares the evidence for the data access committee.

What changes. The methods section writes itself from the record, and the committee approves a release with the evidence in front of it.

Sample sheets from a new collaborating site

Today. A new hospital site joins a study and sends its sample sheet and imaging metadata in its own layout, with its own tissue terms and a free-text consent column.

With Data Workers. Data Ingestion profiles the files, Schema Evolution maps the layout into the model and flags the columns that do not fit, Data Context Wizard proposes ontology mappings with provenance, and a curator approves the ones that are not obvious.

What changes. The site is queryable with the rest of the study in days, and nobody built a one-off script that only one person understands.

For the wider picture on health and life sciences teams, see the health and life sciences data teams overview and our guide to the agentic data platform for healthcare and life sciences.

Measure the product, not just the pipeline

Genomics data leaders are judged on whether scientists reuse what the team builds: cohorts started per month, time from a new dataset landing to its first analysis, and how often an analysis had to be redone because of a data surprise. The Autopilot dashboard shows how much of each data product's upkeep runs on its own and at what token cost, so the team can show where its time now goes: new assays onboarded, better cohorts and fewer redone analyses.

Agents working together: a Monday after an annotation upgrade

Every genomics data team knows the quiet kind of failure: nothing breaks, the workflow reports success, and the results are different. A reference or annotation release is upgraded, a batch is re-run, and the numbers scientists use to make decisions move. This is the same event, handled by Data Workers.

The setup

The warehouse holds sample metadata, variant and expression results tables and extracted imaging metadata, loaded by workflow managers and orchestrated by Airflow, with dbt models on top. It feeds three saved cohorts, a draft figure and a table for a pending submission. Autonomy in Autopilot:

• •Sample sheet mapping, catalog and docs upkeep: L2 today; a candidate for L3 on evidence.
• •Results-table changes that follow a pipeline version, and cohort definitions: L2, held for the bioinformatics lead.
• •Consent scope, data-use terms and external cohort release: stay with the data access committee. Agents prepare evidence only.

The team scheduled one routine: "Compare every results table with its prior release whenever the pipeline or reference version changes."

06:00: the batch reloads

Over the weekend the core upgraded an annotation database release and re-ran batch 14. The results tables reloaded at 06:00 with a new version tag.

06:04: read the run

Connectors agent reads the dbt run history and the version tags on the load and records that the annotation release changed between this batch and the last.

06:10: compare with the prior release

Quality Monitoring agent compares the new tables with what it has learned is normal for them. Classification counts moved in three gene panels, and 212 variant calls changed class. Sample counts, null rates and key uniqueness are unchanged, so the load itself is sound.

06:20: find what depends on it

Data Context & Catalog agent runs a blast radius check: two saved cohorts, one draft figure and one submission table read those results.

06:35: say why

Incident Debugging agent attributes the shift to the release upgrade, not to a sample swap or a failed load, and shows the 212 changed calls beside the release notes the core attached.

06:50: keep both releases queryable

Pipeline Building agent proposes a version-pinned table so the previous release stays queryable, and a diff table the scientists can read. It adds a test that fails if the submission table ever reads an unpinned version.

07:15: check who can see the diff

Data Access & Governance agent checks that the diff table carries nothing beyond what the analysts reading it are already approved to see, and logs the check.

07:30: hold it for the owner

Autonomous Data-Conductor checks the outcomes it can: row counts, key integrity and that every dependent analysis resolves to a pinned version. Because results-table changes sit at L2, Conductor sends one decision to the bioinformatics lead: the full diff, the affected work and the proposed pinning.

08:10: approved

The bioinformatics lead freezes the submission cohort on the old release, moves the two exploratory cohorts to the new one, and changes one of the proposed pins. Conductor applies it and publishes.

09:00: analysts log in

Spellbook Data Catalog pages for the affected tables show the release change, the affected work and the owner's decision. The draft figure carries a note. Nobody files a ticket.

What stays behind

• •Version-pinned results for the submission, with a test that keeps it pinned.
• •A receipt for every change: who proposed it, who approved it and how to roll it back.
• •Updated Spellbook pages for every affected table and cohort.
• •A reusable routine. The next release upgrade runs the same comparison on the day it loads.

Minute by minute

This is an illustrative example. The warehouse is loaded by workflow managers and read by saved cohorts; results-table changes are at L2.

The agents on the job

Data Context Wizard holds one governed definition of each phenotype, tissue, assay and cohort criterion, with provenance, so every analysis and every AI assistant scientists use answers from the same terms. Autonomous Data-Conductor verifies the outcomes that matter before a cohort is used or released. Spellbook Data Catalog (in preview) gives each table, cohort and release package a page that updates when the data does.

What goes on autopilot

Once a domain has earned it, this work runs on its own, and each item removes a problem your team deals with today:

• •Sample sheets and assay exports are mapped on arrival, so a collaborator's extra column does not hold up the cohort build.
• •Results tables are compared with their prior release whenever a pipeline or reference version changes, so a silent shift is a Monday-morning note rather than a reviewer's question.
• •Catalog pages, data dictionaries and lineage are regenerated as the data changes, so the methods behind a cohort are always current. See how an agentic data catalog keeps documentation in step with the data.

Where your people stay in charge

Agents never get blanket access. Each domain has its own autonomy level, set by you, and it only rises when the receipts show the agents are reliable. Everything starts at L1, observing only; reversible upkeep such as documentation, staging fixes and cleanup is usually first to run on its own.

• •Consent scope is decided by a person. Agents show which samples carry which consent and flag conflicts; they never widen a permitted use.
• •Data-use agreements are read and interpreted by the data access committee and legal. Agents attach the recorded terms to each dataset and surface them at release time.
• •External cohort release always goes to the named owner on the data access committee, with lineage, consent evidence and the data dictionary already assembled.
• •Ontology changes that alter a cohort definition go to a named curator.
• •Participant-level access is proposed, approved and logged; the highest-sensitivity tables need two people.

Once a deployment has matured, here is how that ladder plays out across task classes:

Catalog and results-table docs upkeep closes autonomously 85% of the time. For consent scope and external cohort release, autonomous closure is 0% by design: every request is prepared with its evidence and decided by a named person.

What happens when an agent is wrong

• •At L2 propose, nothing changes until a named person approves. The proposal shows the full diff, the blast radius and how to roll it back. Unanswered requests expire and escalate.
• •L3 and L4 are where a domain can go next, on its own review record: acting on its own, with a rollback point on every change. Only the domain's owner raises its level, and if people start reverting the agents' changes, the domain pauses itself.
• •No agent approves or promotes its own work. Autonomy only grows on evidence, domain by domain, and repeated reverts pause the domain.
• •One org-wide stop halts all autonomous dispatch, and every action lands in a tamper-evident audit trail you can export with generate_audit_report.

A wrong ontology mapping is the realistic failure here, and the reason mappings that change a cohort definition stay at L2: the curator sees the term, the evidence and every cohort it would move before anything applies.

How it fits your stack

Data Workers connects to what you already run: workflow managers and orchestrators, the warehouse or lakehouse that holds your results tables, dbt, your catalog and your BI and notebook tools. Each agent is an MCP server, so the same agents work inside Claude Code, Cursor or Codex and in Slack through Spellbook. Nothing migrates.

• •Runs in your infrastructure with your credentials and your model key, including inside your own VPC or on-premises on the Enterprise tier, so participant-level data stays where your governance says it must. A sovereign setting keeps every AI call on models you host yourself.
• •Uses the access you already govern. Agents act under scoped service identities and your existing roles; every grant goes through provision_access and is logged.
• •Reads your catalog's lineage and publishes in the open OpenLineage standard, so other tools can use it. See automated data lineage and the end-to-end lineage transformation.
• •Proposes changes as diffs in your dbt project, orchestrator and warehouse, with blast radius, tests and a rollback point, and applies them only at the autonomy level you set for that domain.
• •Open-source core. The Apache-2.0 community edition ships 11 agents and 160+ MCP tools, so your engineers can read exactly what the agents call.

Governance and regulation

Genetic data is personal data, and under GDPR it is special-category data with a higher bar for processing. Trial and research data add consent, de-identification and access-committee review; in the US, HIPAA rules shape how identifiers are handled in any linked clinical data. Data Workers keeps participant-level tables behind scoped access with enforce_rbac, scans for identifiers in tables and extracted DICOM header fields with scan_pii, attaches recorded consent and data-use terms to each dataset, and records every access and change in a tamper-evident trail. The standard Data Use Ontology from GA4GH gives teams a shared vocabulary for dataset permissions, and the agents can carry whatever codes your committee uses next to the data. Your privacy, legal and access-committee teams decide sufficiency.

Data Workers doesn't make you compliant; your controls, people and assessors do. It supplies controls and evidence wherever agents touch data. Not legal advice. For the longer treatment, see how Data Workers helps with HIPAA, GDPR and the EU AI Act.

The alternatives genomics, imaging and multi-omics data teams weigh

Build it yourselves. A strong team can wire a coding agent to MCP servers for the warehouse and dbt. The work is everything around it: routing approvals to the right owner, blast radius, rollback, receipts, and current context about every other system.

Workflow platforms, LIMS and analysis environments. Platforms such as Terra, DNAnexus and your LIMS run the pipelines and track the samples, and they should keep doing it. The upkeep of the tables downstream of them, the ontologies, the cohort definitions and the access rules usually falls between tools. Data Workers works on that layer and leaves the analysis where it runs.

Observability, catalogs and point tools. Observability alerts, catalogs document, and quality tools test. Data Workers takes their signals, diagnoses, proposes the fix, verifies it and leaves the receipt. It reads from the catalog you already run rather than replacing it.

In this space specifically. Clinical trial platforms and commercial RWD tools handle their own domains; see the clinical and R&D data and pharma commercial and RWD playbooks. Data Workers keeps the research-data layer under analyses correct and traceable.

The case for your CTO

What this replaces. Spreadsheet-driven sample mapping, per-study ontology fixes, results that change without anyone noticing, and a hand-built paper trail for every cohort.

What your team gets back. A genomics data team that ships reusable cohorts and clean results tables instead of reconciling spreadsheets and rebuilding last year's analysis from memory.

Integration effort. Connecting is configuration, not a migration. The agents attach over MCP to the warehouse, orchestrator, dbt project and source systems you already run, under the roles you already govern, inside your own infrastructure. Your engineers can read every tool the agents call in the open-source core.

Risk and control. Autonomy is set per domain. Everything starts at L1, observing only; reversible work moves up only when the receipts justify it; consent scope, data-use agreements and cohort release stay with a named owner; and one org-wide stop halts all autonomous work.

Why now. AI models trained on, and AI assistants querying, research data multiply the cost of an unseen version change or a cohort nobody can reproduce. The teams that can show lineage and provenance for every dataset are the ones that can use their data for AI at all. See the AI and ML platform data playbook for the model side.

The first win. Pipeline-version drift checks and cohort documentation on one assay family. The pilot below lays out the scope, what you see each week and the cost.

The climb: from assisted to autonomous to agentic

Our thesis is that the data platform climbs the way analytics did, in stages, and that the hard part of data work was never knowing what to do but doing it. For genomics, imaging and multi-omics data teams, the climb looks like this:

• •Assisted. Agents work alongside your team. They read the whole stack, show what needs doing and propose every fix with a receipt; consent scope, data-use agreements and cohort release always go to a named owner.
• •Autonomous, under your governance. Routine work runs on its own: sample sheets and assay exports are mapped on arrival, results tables are compared with their prior release whenever a version changes, and catalog pages and data dictionaries stay current. Your team governs the agents instead of doing the upkeep.
• •The agentic enterprise. A genomics data team that ships reusable cohorts and clean results tables instead of reconciling spreadsheets. People set direction, and the platform carries it out.

You choose the altitude, every step is visible and reversible, and the pilot below is the first rung.

The pilot: how genomics, imaging and multi-omics data teams start

The pilot is how your team proves the value on its own data before committing.

• •What we connect. The warehouse or lakehouse that holds one assay family's results tables, the sample metadata, dbt, the orchestrator and your catalog, over MCP, inside your infrastructure. Nothing migrates, and raw files stay in object storage.
• •Scope. Pipeline-version drift checks and cohort documentation on one assay family, at L1 and L2.
• •First week. Agents run at L1 across the scoped domain. They map every table, definition and lineage path, and show the version comparisons and cohort documentation they would have produced, with receipts, so your team sees the value before anything changes.
• •By week two. Sample sheet mapping and catalog and docs upkeep start running on reversible work. Results-table changes and cohort definitions stay at L2 with a named approver; consent scope and release stay with the data access committee.
• •At day 30. We review the impact metrics against your baseline:

• •Cohort rebuilt as published: from about 40 hours to 2 hours
• •Result shift traced to a pipeline change: from about 24 hours to 30 minutes
• •Consent evidence for a release assembled: from about 12 hours to 1 hour
• •the share of catalog and docs upkeep closed autonomously, targeting 85% as the deployment matures

• •What you decide. Whether to expand to the next assay family or biobank. The pilot fee is credited in full against the first year.
• •Cost. Start with a pilot ($7,500 one-time, a 6 to 12 week pilot, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.

The pilot is the start of a longer curve:

By month 12, around 38% of agent actions run fully autonomously and manual work falls well below where it starts, while consent scope, data-use terms and external release stay with a named owner.

FAQ

Do the agents run variant calling or image analysis? No. They work on the metadata, catalog, lineage, quality and access layer and on the tabular results in your warehouse or lakehouse. Pipelines and analysis stay where they run.

Can they read BAM, VCF or pixel data? No. They read what your pipelines write into tables, plus run records and version tags. Raw files stay in object storage.

Who decides whether a cohort can go to an external partner? The data access committee. Agents assemble lineage, consent evidence and the data dictionary; a named person decides.

Does it handle DICOM? It works on DICOM header and digital pathology metadata once extracted into tables: profiling fields, flagging identifier-bearing fields, mapping them into your model and keeping lineage. The standard itself is maintained by the DICOM standards committee.

Where do the agents run? In your infrastructure, with your credentials and model key. The hosted Conductor sees workflow metadata only.

Do we have to migrate anything? No. Your warehouse, dbt project, orchestrator, catalog and BI stay where they are.

Sources

• •GA4GH, Data Use Ontology (DUO): https://www.ga4gh.org/product/data-use-ontology-duo/
• •NIH, Genomic Data Sharing Policy: https://sharing.nih.gov/genomic-data-sharing-policy
• •Regulation (EU) 2016/679 (GDPR), Article 9 on special categories of personal data: https://eur-lex.europa.eu/eli/reg/2016/679/oj
• •DICOM Standard: https://www.dicomstandard.org/
• •HHS, Guidance on de-identification of protected health information: https://www.hhs.gov/hipaa/for-professionals/special-topics/de-identification/index.html
• •Data Workers pricing: https://dataworkers.io/pricing/