The agentic market and reference data team: every price, rate and security master verified before it reaches a valuation
Every price, rate and security master verified before it reaches a valuation. That is the bar for market and reference data teams once agents run the data back office, and it is reachable with the estate you already have. Here is how it works, use case by use case, and what changes compared with today.
Key takeaways
- •Who this is for. The team that owns the vendor feeds every valuation depends on: prices, curves, FX rates, corporate actions, the security master and counterparty reference data.
- •The problem today. Morning email threads about which vendor price is right, NAV re-strikes after vendor layout changes and annual entitlement spreadsheets.
- •What changes. Every vendor file is profiled the minute it lands, so a shifted column is caught before anything values off it. Prices are compared across vendors by your golden-source rules, so only real exceptions reach a person.
- •What stays with your people. Changes to NAV, risk or client-reporting inputs: valuations carry accountability, so the pricing lead approves.
- •The result. A market data team that owns golden-source policy instead of babysitting files.
The PRD for agentic market and reference data teams
This is the requirements document we would write with you for your team. Everything below expands on it.
| Who | Market data, reference data, pricing and security master teams at banks, asset managers and brokers |
| The job | Every price, rate and security master verified before it reaches a valuation. |
| Environment | Snowflake, Databricks or BigQuery; Airflow or Control-M; dbt; vendor SFTP and APIs from Bloomberg, Refinitiv/LSEG, ICE and S&P; an EDM or security master. |
| Constraints | Changes to NAV, risk or client-reporting inputs always go to a named approver. BCBS 239 asks banks for accurate, complete and timely risk data with clear lineage, and vendor licenses restrict who may see which feed. |
| Success is measured by | Lineage for an audited price: from about 8 hours to seconds; License usage report: from about 2 weeks to ~1 h; Vendor layout change mapped: from about 2 days to 15-30 min; catalog and vendor docs closed autonomously 88% of the time. |
| Owned by | A Head of Market Data, with domain owners as named approvers. |
| Pilot scope | Freshness SLAs and cross-vendor checks on the pricing feeds, at L2 propose, for one quarter. |
What going fully agentic and autonomous means for market and reference data teams
- •Put your data pipeline on autopilot. Every vendor file is profiled the minute it lands, so a shifted column is caught before anything values off it.
- •Run the data back office without the busywork. The work that fills the week today moves to agents: morning email threads about which vendor price is right, NAV re-strikes after vendor layout changes and annual entitlement spreadsheets.
- •Scale data without scaling headcount. Prices are compared across vendors by your golden-source rules and entitlement grants and usage are logged continuously, so the data you manage grows without the team having to.
- •Get the plumbing off your plate. A market data team that owns golden-source policy instead of babysitting files.
Who this is for
The team that owns the vendor feeds every valuation depends on: prices, curves, FX rates, corporate actions, the security master and counterparty reference data. It usually sits in operations or a chief data office, and its consumers are fund accounting, risk, trading and client reporting. Titles include Head of Market Data, Reference Data Product Owner, Pricing Data Lead and Security Master Manager.
The shift
Firms license data from a dozen vendors, each with its own formats, restatements and outage patterns, and every downstream number inherits their mistakes. Licensing costs keep rising, and AI copilots now quote prices and exposures directly to portfolio managers. The work of keeping vendor data clean, mapped, entitled and traceable is enormous and repeatable, which makes it the clearest case in the bank for an agentic back office.
The product you ship
- •Golden-source prices, curves and FX rates with an audit trail per value
- •The security master and corporate actions every system reads
- •Vendor entitlement and licensing controls
- •Data products for valuation, risk and client reporting
Where the week goes
- •A vendor changes a file layout and a currency lands in the wrong column.
- •A late file delays the pricing cut-off.
- •Corporate actions arrive from two vendors and disagree.
- •Entitlement reviews and license audits are done by hand.
- •Every downstream team asks where a price came from.
Add it up and upkeep is most of the team's week. Here is how a typical week splits today, and once agents own the back office:
Time spent on new data products moves from 16% of the week to 50%. Most of the rest becomes review: people steer and approve agent work instead of doing it by hand.
Then and now
| Work | Today | With Data Workers |
|---|---|---|
| A vendor layout change | Found when a NAV looks wrong | Caught on arrival, mapped and verified before use |
| Two vendors disagree on a price | Resolved by hand before cut-off | Compared automatically, golden source chosen by rule, exceptions to a person |
| A late file | Cut-off missed | Freshness SLA tracked, fallback source proposed |
| Where did this price come from? | Hours of tracing | Lineage from value to vendor file in one query |
| License and entitlement audits | Spreadsheets once a year | Access logged and scoped continuously |
These are the impact metrics we move for market and reference data teams:
- •Lineage for an audited price: from about 8 hours to seconds
- •License usage report: from about 2 weeks to ~1 h
- •Vendor layout change mapped: from about 2 days to 15-30 min
Across the routine work in this domain:
The biggest change is License usage report: from about 2 weeks to ~1 h.
Use cases, in depth
The currency in the wrong column
Today. A vendor adds a column and 140 positions price in the wrong currency; fund accounting finds it in a NAV variance two hours later.
With Data Workers. Data Ingestion profiles the file on arrival, Schema Evolution maps the new column, Quality Monitoring checks prices tie to the vendor's own totals, and Conductor holds the NAV input change at L2 for the pricing lead.
What changes. The break never reaches the NAV.
Two vendors, one corporate action
Today. A stock split arrives from two vendors with different ratios; ops spends the morning deciding which to trust.
With Data Workers. Quality Monitoring flags the disagreement, Data Context Wizard applies the golden-source rule the team defined, and the exception goes to one person with both records side by side.
What changes. Corporate actions are resolved before positions update.
License and entitlement control
Today. Vendor audits ask who used which feed; nobody can say.
With Data Workers. Data Access & Governance scopes entitlements with provision_access, logs every grant, and generate_audit_report exports usage per vendor.
What changes. License audits take an afternoon, not a quarter.
Agents working together: A currency in the wrong column, fixed before cut-off
Minute by minute
This is an illustrative example. An asset manager loads vendor price files into Snowflake through Airflow; a dbt model builds the golden-source price table read by fund accounting at 10:30. The pricing domain is at L2 propose.
| Time | Who | What happens |
|---|---|---|
| 06:40 | Vendor SFTP | A price file lands with a new column; the currency code shifts one position |
| 06:41 | Airflow + Snowflake | The DAG loads it; 140 fixed-income rows carry the wrong currency |
| 06:42 | Schema Evolution agent | detect_schema_change flags the layout change; run_quality_check fails the tolerance test |
| 06:45 | Data Context & Catalog agent | blast_radius_analysis: the 10:30 NAV input and two risk reports |
| 06:49 | Pipeline Building agent | Mapping and model diff proposed with tests and a rollback point; create_jira_sm_ticket opens the ticket |
| 07:05 | Spellbook Data Catalog | The pricing lead reviews the diff and approves |
| 07:08 | Autonomous Data-Conductor, via Airflow | Reload and dbt rebuild; prices tie to vendor totals |
| 07:10 | Spellbook Data Catalog | Receipt written to the ticket; the lead closes it |
| 10:30 | Fund accounting | NAV inputs read verified prices |
Without the agents, the NAV variance surfaces at 10:30 and the strike slips. With them, one person approves one diff before 7:10.
The agents on the job
| Agent | What it does for you | Tools it calls |
|---|---|---|
| Data Ingestion | Profiles every vendor file on arrival and detects late or restated deliveries | get_connector_status, trigger_airflow_dag, register_kafka_schema |
| Schema Evolution | Maps vendor layout changes and versions each feed contract | detect_schema_change, validate_schema_compatibility, generate_migration, rollback_migration |
| Quality Monitoring | Price tolerance, stale-price and cross-vendor comparison checks | set_sla, run_quality_check, get_anomalies, create_quality_tests_for_pipeline |
| Incident Debugging | Traces a bad valuation to the file and field behind it | diagnose_incident, get_root_cause, prepare_backfill, execute_backfill |
| Data Access & Governance | Scopes vendor entitlements and logs usage for license audits | provision_access, check_policy, request_governance_review, generate_audit_report |
| Data Context & Catalog | Keeps golden-source rules, lineage and vendor terms current | trace_cross_platform_lineage, blast_radius_analysis, define_business_rule, generate_documentation |
Data Context Wizard keeps one governed graph of definitions, lineage and owners that every agent and person answers from. Autonomous Data-Conductor runs each job from detection to a verified result. Spellbook Data Catalog is where your team approves, audits and rolls back, and asks questions in plain English.
What goes on autopilot
Once a domain has earned it, this work runs on its own, and each item removes a problem your team deals with today:
- •Every vendor file is profiled the minute it lands, so a shifted column is caught before anything values off it.
- •Prices are compared across vendors by your golden-source rules, so only real exceptions reach a person.
- •Entitlement grants and usage are logged continuously, so a vendor audit takes an afternoon.
Where your people stay in charge
Agents never get blanket access. Each domain has its own autonomy level, set by you, and it only rises when the receipts show the agents are reliable. Everything starts at L1, observing only; reversible upkeep such as reruns, staging fixes, documentation and cleanup is usually first to run on its own.
- •Changes to NAV, risk or client-reporting inputs always goes to a named approver: valuations carry accountability, so the pricing lead approves.
- •Golden-source policy and vendor contracts: these stay with people.
Once a deployment has matured, here is how that ladder plays out across task classes:
Catalog and vendor docs closes autonomously 88% of the time. For NAV and risk inputs, autonomous closure is 0% by design.
What happens when an agent is wrong
- •At L2 propose, nothing changes until a named person approves. The proposal shows the full diff, the blast radius and how to roll it back. Unanswered requests expire and escalate.
- •At L3 act, reversibly, every change keeps a rollback point, and Conductor verifies the downstream number after the change. If verification fails, the change is reversed and the domain drops a level.
- •No agent approves or promotes its own work. Autonomy only grows on evidence, per repair class, and repeated errors shrink scope automatically.
- •One org-wide stop halts all autonomous dispatch, and every action lands in a hash-chained audit trail you can export with generate_audit_report.
How it fits your stack
Data Workers connects to what you already run: Snowflake, Databricks or BigQuery; Airflow or Control-M; dbt; vendor SFTP and APIs from Bloomberg, Refinitiv/LSEG, ICE and S&P; an EDM or security master. Each agent is an MCP server, so the same agents work inside Claude Code, Cursor or Codex and in Slack or Teams through Spellbook. Nothing migrates.
- •Runs in your infrastructure with your credentials and your model key. The hosted Autonomous Data-Conductor sees workflow metadata only, never rows.
- •Uses the access you already govern. Agents act under scoped service identities and your existing roles; every grant goes through provision_access and is logged.
- •Proposes changes as diffs in your dbt project, orchestrator and warehouse, with blast radius, tests and a rollback point, and applies them only at the autonomy level you set for that domain.
- •Open-source core. The Apache-2.0 community edition ships 11 agents and 160+ MCP tools, so your engineers can read exactly what the agents call.
Governance and regulation
BCBS 239 asks banks for accurate, complete and timely risk data with clear lineage, and vendor licenses restrict who may see which feed. Data Workers supplies lineage from every value to its source file, scoped access with provision_access, and a hash-chained audit trail exported with generate_audit_report. Your control owners decide what satisfies each requirement.
Data Workers doesn't make you compliant; your controls, people and assessors do. It supplies controls and evidence wherever agents touch data. Not legal advice.
The alternatives market and reference data teams weigh
Build it yourselves. A strong team can wire a coding agent to MCP servers for the warehouse and dbt. The work is everything around it: routing approvals to the right owner, blast radius, rollback, receipts, and current context about every other system.
Observability, catalogs and point tools. Observability alerts, catalogs document, and quality tools test. Data Workers takes their signals, diagnoses, proposes the fix, verifies it and leaves the receipt.
Platform-native agents. Snowflake made its managed MCP server and Cortex Agents generally available on Nov 4, 2025, and Databricks made Genie One generally available on Jun 16, 2026. They are excellent inside their platform; this work crosses the source systems, orchestrator, warehouse and BI, and Data Workers works across all of them.
In this space specifically. EDM platforms (GoldenSource, Markit EDM) hold the golden copy; Data Workers keeps the feeds into it and the systems after it correct, and works alongside them.
The case for your CTO
What this replaces. Morning email threads about which vendor price is right, NAV re-strikes after vendor layout changes and annual entitlement spreadsheets.
What your team gets back. A market data team that owns golden-source policy instead of babysitting files.
Integration effort. Connecting is configuration, not a migration. The agents attach over MCP to the warehouse, orchestrator, dbt project and source systems you already run, under the roles you already govern, inside your own infrastructure. Your engineers can read every tool the agents call in the open-source core.
Risk and control. Autonomy is set per domain. Everything starts at L1, observing only; reversible work moves up only when the receipts justify it; changes to NAV, risk or client-reporting inputs stay at L2 with a named approver; and one org-wide stop halts all autonomous work.
Why now. Vendor costs and data volumes keep rising, and copilots now quote prices directly to the front office.
The first win. Freshness SLAs and cross-vendor checks on the pricing feeds, at L2 propose, for one quarter. The pilot below lays out the scope, what you see each week and the cost.
The climb: from assisted to autonomous to agentic
Our thesis is that the data platform climbs the way analytics did, in stages, and that the hard part of data work was never knowing what to do but doing it. For market and reference data teams, the climb looks like this:
- •Assisted. Agents work alongside your team. They read the whole stack, show what needs doing and propose every fix with a receipt; changes to NAV, risk or client-reporting inputs always go to a named approver.
- •Autonomous, under your governance. Routine work runs on its own: every vendor file is profiled the minute it lands, prices are compared across vendors by your golden-source rules, and entitlement grants and usage are logged continuously. Your team governs the agents instead of doing the upkeep.
- •The agentic enterprise. Within a few years, market data teams will run a fully autonomous vendor layer: every file profiled, compared and entitled without a person in the loop, with humans owning golden-source policy and vendor strategy. The teams that get there first will cut license spend and become the fastest source of truth in their firm.
You choose the altitude, every step is visible and reversible, and the pilot below is the first rung.
The pilot: how market and reference data teams start
The pilot is how your team proves the value on its own data before committing.
- •What we connect. Snowflake, Databricks or BigQuery; Airflow or Control-M; dbt; vendor SFTP and APIs from Bloomberg, Refinitiv/LSEG, ICE and S&P; an EDM or security master, over MCP, inside your infrastructure. Nothing migrates.
- •Scope. Freshness SLAs and cross-vendor checks on the pricing feeds, at L2 propose, for one quarter.
- •First week. Agents run at L1 across the scoped domain. They map every pipeline, definition and lineage path and show the fixes they would have made, with receipts, so your team sees the value before anything changes.
- •By week two. Every vendor file is profiled the minute it lands and prices are compared across vendors by your golden-source rules start running on reversible work. Changes to NAV, risk or client-reporting inputs stay at L2 with a named approver.
- •At day 30. We review the impact metrics against your baseline:
- •Lineage for an audited price: from about 8 hours to seconds
- •License usage report: from about 2 weeks to ~1 h
- •Vendor layout change mapped: from about 2 days to 15-30 min
- •the share of catalog and vendor docs closed autonomously, targeting 88% as the deployment matures
- •What you decide. Whether to expand to the next domain. The pilot fee is credited in full against the first year.
- •Cost. Start with a pilot ($7,500 one-time, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.
The pilot is the start of a longer curve:
By month 12, more than 38% of agent actions run fully autonomously while NAV and risk inputs stay with a named approver.
FAQ
Does it replace our EDM? No. It keeps the feeds into the EDM and every consumer after it correct, and it can read golden-source rules from it.
Can it see vendor data we are not licensed to share? Agents use the access you grant. Entitlements are scoped with provision_access and every read is logged.
What about prices that look wrong but are real? Quality checks propose; a pricing analyst confirms. Tolerances and exceptions are rules the team owns.
Where do the agents run? In your infrastructure, with your credentials and model key. The hosted Conductor sees workflow metadata only.
Do we have to migrate anything? No. Your warehouse, dbt project, orchestrator, catalog and BI stay where they are.
Sources
- •JPMorgan Chase, Chief Data & Analytics Office careers page: https://www.jpmorganchase.com/careers/explore-opportunities/programs/data-analytics-opportunities
- •SiliconANGLE, How JPMorgan Chase runs a data mesh on AWS: https://siliconangle.com/2021/07/10/new-era-data-deep-look-jpmorgan-chase-runs-data-mesh-aws-cloud/
- •Data Workers pricing: https://dataworkers.io/pricing/
- •Data Workers open-source repository (tool registrations): https://github.com/DataWorkersProject/dataworkers-claw-community
- •Snowflake, managed MCP server and Cortex Agents GA release note (Nov 4, 2025): https://docs.snowflake.com/en/release-notes/2025/other/2025-11-04-cortex-agents-mcp
- •Databricks, Genie One launch (Jun 16, 2026): https://www.databricks.com/company/newsroom/press-releases/databricks-launches-genie-one-all-new-agentic-coworker-every-team