Agentic recommendation data: a guide for personalization data teams

Key takeaways

• •Who this is for. The team that supplies a recommender or personalization product with its data: interaction logs, item and content catalogs, user and item features, ranking training sets and experiment data.
• •The problem today. A mobile release renames a click field, a catalog merge retires ten thousand item IDs, a stream delivers events twice, and the offline features the model trained on quietly stop matching the online features it serves. Nobody notices until click-through falls, and then the investigation takes days.
• •What changes. Agents read the event stream, the catalog and the feature pipelines continuously, so a changed event is caught on the release that changed it, the affected features and training sets are listed within minutes and the bad window is held out of tonight's training run.
• •What stays with your people. Which model reaches production, which experiment ships, and what consent allows. Those calls go to a named approver with the evidence attached.
• •The result. A personalization data team that spends its week on new signals and better candidate sets, not on explaining why the ranker got worse on Tuesday.

The PRD for agentic personalization and recommendations data teams

This is the requirements document we would write with you for your team. Everything below expands on it.

What going fully agentic and autonomous means for personalization and recommendations data teams

• •Put your data pipeline on autopilot. Event schema changes, duplicate and late events and catalog changes are mapped, repaired and verified the day they happen, so the nightly training run never reads a window that was already broken.
• •Run the data back office without the busywork. The work that fills the week today moves to agents: tracing a click-through drop back to a renamed field, rebuilding training sets by hand after a catalog change and documenting features nobody remembers writing.
• •Scale data without scaling headcount. Every new surface, locale and experiment adds events, features and metrics. Agents keep them described, tested and in agreement, so the data you manage grows without the team having to.
• •Get the plumbing off your plate. A personalization data team that ships new signals and candidate generators instead of arbitrating between offline and online numbers.

Who this is for

The team that sits between product instrumentation and the models that rank. At a streaming or media company it supplies the home feed and the next-episode ranker. At a marketplace or retailer it supplies search ranking, product recommendations and email personalization. At a consumer app it supplies notification timing and content feeds. In every case the team owns a mix of streams, tables and features, and answers to both machine learning engineers who consume them and product managers who run experiments on top of them.

The titles: Head of Personalization Data, Director of Recommendation Platform, Staff Data Engineer on a ranking team, ML Feature Platform Lead, and Experimentation Data Product Manager. These leaders usually run several surfaces at once, each with its own models, features and tests, and a shared pool of event data that all of them depend on.

This playbook is one of the product and usage data teams pages; the rest of that group covers the neighbouring teams you share data with.

The product you ship

• •Interaction data: impressions, clicks, plays, purchases and dwell time, deduplicated, ordered and joined to what was actually shown, as a governed data product.
• •Item and content catalogs: versioned items, categories and attributes, with merges, retirements and relaunches that do not orphan history.
• •User and item features: the values ranking models read, defined once and computed the same way offline and online.
• •Candidate generation and ranking training data: point-in-time correct training sets with labels, exposures and negatives that can be rebuilt on demand.
• •Experiment data: assignment, exposure and metric tables that make a readout believable.
• •Consent-aware data: interaction history and features that honour opt-outs and deletion requests across every copy.

Where the week goes

• •Instrumentation drifts. Product teams rename fields, change when an impression fires or ship a new client without telling anyone. The first sign is a model metric, days later.
• •Events arrive wrong. Late, duplicated and out-of-order events inflate counts and corrupt session windows.
• •Catalogs change under the models. Items merge, split, expire and get relaunched under new IDs, and embeddings and candidate sets point at items that no longer exist.
• •Training and serving disagree. A feature computed in a batch job and the same feature computed from the stream drift apart. The feature store is meant to prevent this, and it only prevents what it is told about.
• •Experiments depend on clean assignment. A readout is only as good as the exposure data under it, and nobody checks that until the result looks odd.
• •Consent and personal data are everywhere. Behavioural history sits in logs, features, training snapshots and embeddings, and an opt-out has to reach all of them.
• •Nobody can say what trained the model. When a ranker misbehaves, reconstructing which data fed which model version is a project of its own.

Add it up and upkeep is most of the team's week. Here is how a typical week splits today, and once agents own the back office:

Time spent on new data products moves from 18% of the week to 48%. Most of the rest becomes review: people steer and approve agent work instead of doing it by hand.

Then and now

These are the impact metrics we move for personalization and recommendations data teams:

• •Broken event traced to its release: from about two days to minutes
• •Training set rebuilt after a catalog change: from about a day to under an hour
• •Training and serving skew traced to its cause: from about three days to under an hour

Across the routine work in this domain:

The biggest change is skew investigation: from about three days to under an hour. Work that waited for someone with the right context finishes in minutes, because agents detect, trace and verify in one pass.

Use cases, in depth

Instrumentation drift on a release

Today. A client release renames a field or fires an impression twice. Click-through shifts, the feature pipelines keep running and the training job reads the damage that night.

With Data Workers. The Connectors agent registers the new event shape as it appears, Schema Evolution flags the breaking change, Data Context & Catalog lists every feature, dataset and metric downstream, and Quality Monitoring sizes the duplicate or missing events. The fix lands in staging and the affected window is held out of training. The pattern is covered in more depth on the instrumentation QA page.

What changes. The release that broke the event is the first thing in the report, not the last thing anyone finds.

Training and serving skew

Today. Offline evaluation looks fine and online quality falls. The team compares feature values by hand, one at a time, until someone finds the one computed differently in the batch job and the stream. Google's Rules of Machine Learning names this gap between how a model performs in training and how it performs in serving as training-serving skew.

With Data Workers. Data Context & Catalog traces each suspect feature to its sources and transforms. MLOps & Models compares the feature's training and serving distributions and shows when they began to diverge and which model version was hit. Quality Monitoring adds a standing parity test so the same mismatch cannot return unnoticed. Your feature store keeps serving the values; the agents keep watch over whether they still mean what the model learned. More on that division of labour in You're on Feast.

What changes. Skew is a finding with a date and a cause, not a three-day investigation.

An experiment readout you can trust

Today. An experiment finishes, the metric moves and the room argues about whether to believe it. Late assignment, duplicate exposures and a metric table that changed mid-test are found after the meeting.

With Data Workers. Before the readout, Quality Monitoring checks assignment and exposure data, Schema Evolution confirms the metric definitions did not change under the test, and Data Context & Catalog records which data version each arm read. MLOps & Models keeps the experiment records tied to the model versions behind them. The full pattern is on the experiment readout trust page.

What changes. The readout starts with data that has already been checked, so the discussion is about the result.

Measure the product, not just the pipeline

Personalization leaders are judged on whether the ranker improved and whether experiments were believable, not on table counts. The Autopilot dashboard shows how much of each data product's upkeep runs on its own and at what token cost, so the team can show leadership where its time now goes: new signals, better candidate sets and faster experiment cycles.

Agents working together: a bad release, caught before tonight's training run

Every personalization team knows this morning. A mobile release goes out, nothing alerts, and three days later the home-feed ranker is quietly learning from a broken signal. This is the same release handled by Data Workers.

The setup

The interaction pipeline runs on Kafka, lands in Snowflake and is transformed with dbt under Airflow. It feeds a home-feed ranker, a set of user and item features served online, a nightly training job and an experiment platform with several tests running. Autonomy in Autopilot:

• •Staging fixes and event schema mapping: L2 today; a candidate for L3 on evidence.
• •Feature definitions and training sets of the production ranker: L2, held for the ranking lead.
• •Consent and personal data: L2, held for the privacy lead.
• •Model promotion and experiment ship decisions: L0.

The team scheduled one mission a week earlier: "Keep the home-feed interaction data and features in parity and flag any release that changes them."

09:12, the release rolls out

The mobile team starts a staged rollout. The new build renames the item field on the carousel click event and, because of a scroll-restore change, fires impressions twice.

09:14, the stream looks different

Real-Time Streaming agent sees impression volume per session jump on one app build while clicks hold steady. It hands the observation on instead of waiting for a daily report.

09:15, the change is named

Connectors agent registers the new event shape with register_stream_schema. Schema Evolution agent detects the renamed field and works out the mapping back to the established item identifier.

09:20, the blast radius

Data Context & Catalog agent lists what depends on the changed events: the click-through feature, two session features, the nightly training set and two experiment metrics. It names the owner of each.

09:30, prove the damage

Quality Monitoring agent confirms that impressions per session are inflated on the new build and that click-through rate is therefore understated for that cohort. It sizes the affected window and the share of sessions involved.

09:40, fix it in staging

Schema Evolution agent maps the renamed field in staging and applies a duplicate-impression rule to the cleaned layer, keeping the raw events untouched. It is proposed as a reversible change, with a rollback point, for the owner to approve.

09:55, protect the model

MLOps & Models agent compares the online and offline feature values for the affected window and confirms they now disagree. It proposes to exclude the window from tonight's training set until the client is fixed and to rebuild the training set from the corrected layer.

10:20, one decision

Autonomous Data-Conductor verifies that the corrected staging numbers reconcile with the raw events and that the experiment metrics move back into range. Because the production ranker's training set sits at L2, Conductor sends one decision to the ranking lead: the window to exclude, the diff to the training set and the experiments affected.

10:35, approved

The ranking lead approves the exclusion and asks for one experiment to be annotated so its readout notes the affected window. Conductor applies both.

11:00, the experiments are protected

The affected tests carry an annotation. The mobile team gets a precise note on what the new build changed. Nobody has opened a ticket.

What stays behind

• •A named record of the changed event, the window it affected and the fix.
• •A standing parity test between the stream and the batch feature for the affected signal.
• •Updated Spellbook pages for every affected feature and dataset.
• •A reusable mission. The next release runs the same checks, and the routine parts can move to L3 as the receipts earn it.

Minute by minute

This is an illustrative example. The interaction pipeline runs on Kafka, Snowflake and dbt under Airflow; the ranker trains nightly and the features are served online. Training sets and consent are at L2.

The agents on the job

Data Context Wizard holds one governed definition of an impression, a click, a session, an exposure and every feature the ranker reads. Dashboards, experiment readouts and the feature pipelines all answer from the same definitions, so the number in the readout and the number in the model's training set agree. Autonomous Data-Conductor verifies the figures that matter before a training run and before a readout. Spellbook Data Catalog makes each interaction table, feature and experiment dataset discoverable, with its owner and consumers, in plain English.

What goes on autopilot

Once a domain has earned it, this work runs on its own, and each item removes a problem your team deals with today:

• •Event schema changes are mapped into staging the day they happen, so a renamed field never silently becomes a null feature.
• •Duplicate and late events are detected and repaired in the cleaned layer, so click-through and session features stop swinging with client bugs.
• •Feature, dataset and metric documentation stays current, so nobody has to ask what a feature means or which tables feed it.
• •Parity between batch and stream features is tested daily, so skew shows up as a failed test, not a quality drop.

Where your people stay in charge

Agents never get blanket access. Each domain has its own autonomy level, set by you, and it only rises when the receipts show the agents are reliable. Everything starts at L1, observing only; reversible upkeep such as staging fixes, reruns, documentation and cleanup is usually first to run on its own.

• •Training sets and feature definitions for production rankers go to a named approver: a changed training set changes what the model learns, so the ranking lead approves each one with the diff and the affected experiments.
• •Consent and personal data go to the privacy lead: opt-outs, deletions and any change to what a feature may contain.
• •Model promotion and experiment ship decisions stay entirely with people.

Once a deployment has matured, here is how that ladder plays out across task classes:

Catalog and docs upkeep closes autonomously 88% of the time. For ranking model promotion and ship calls, autonomous closure is 0% by design: every recommendation arrives with evidence, and a named person decides.

What happens when an agent is wrong

• •At L2 propose, nothing changes until a named person approves. The proposal shows the full diff, the blast radius and how to roll it back. Unanswered requests expire and escalate.
• •L3 and L4 are where a domain can go next, on its own review record: acting on its own, with a rollback point on every change. Only the domain's owner raises its level, and if people start reverting the agents' changes, the domain pauses itself.
• •No agent approves or promotes its own work. Autonomy only grows on evidence, domain by domain, and repeated reverts pause the domain.
• •One org-wide stop halts all autonomous dispatch, and every action lands in a tamper-evident audit trail you can export with generate_audit_report.

For a recommender the worst error is a silent one: a window wrongly excluded or a field wrongly mapped. That is why raw events are never rewritten, staging changes are reversible and any change to a production training set is a proposal.

How it fits your stack

Data Workers connects to what you already run: Kafka or a cloud streaming service; Snowflake, Databricks or BigQuery; dbt; Airflow or Dagster; a feature store such as Feast or Tecton; MLflow or a similar tracker for experiments and models; and your experimentation platform. Each agent is an MCP server, so the same agents work inside Claude Code, Cursor or Codex and in Slack or Teams through Spellbook. Nothing migrates.

• •Runs in your infrastructure with your credentials and your model key. The hosted Autonomous Data-Conductor sees workflow metadata only, never rows.
• •Uses the access you already govern. Agents act under scoped service identities and your existing roles; every grant goes through provision_access and is logged.
• •Works beside your feature store and trackers. The feature store serves features and the tracker keeps runs; Data Workers checks the data under both. See You're on Feast for how the two divide the job, and Inside the MLOps & Models Agent for the model-side view.
• •Proposes changes as diffs in your dbt project, orchestrator and warehouse, with blast radius, tests and a rollback point, and applies them only at the autonomy level you set for that domain.
• •Open-source core. The Apache-2.0 community edition ships 11 agents and 160+ MCP tools, so your engineers can read exactly what the agents call.

For the streaming side specifically, Inside the Real-Time Streaming Agent covers what the agent does with lag, shape and delivery problems.

Governance and regulation

Behavioural data is personal data. Interaction history, the features computed from it and the training snapshots built on it all fall under the consent and deletion rules you operate under, and an opt-out has to reach every copy. Data Workers finds personal data in logs and feature tables with scan_pii, applies role-based rules with check_policy, holds changes to consent-related data for the privacy lead and records every change in a tamper-evident trail. Your consent platform stays the source of truth for what a person agreed to; the agents check that your pipelines honour it, as described in You're on OneTrust.

Data Workers doesn't make you compliant; your controls, people and assessors do. It supplies controls and evidence wherever agents touch data. Not legal advice.

The alternatives personalization and recommendations data teams weigh

Build it yourselves. A strong team can wire a coding agent to MCP servers for the warehouse and dbt. The work is everything around it: routing approvals to the ranking lead and the privacy lead, blast radius across features and experiments, rollback, receipts, and current context about every other system.

Feature stores, trackers and point tools. Feature stores serve features, trackers record runs, observability tools alert and quality tools test. Feast documents point-in-time correct joins for building training sets without leaking future data, and that is the feature store doing its job. Data Workers takes their signals and checks the data beneath them, diagnoses a break across the stream, the catalog and the features, proposes the fix, verifies it and leaves the receipt. For the data-quality side of ML, see data quality for ML.

Platform-native agents. Snowflake made its managed MCP server and Cortex Agents generally available on Nov 4, 2025, and Databricks made Genie One generally available on Jun 16, 2026. They are excellent inside their platform; this work crosses the client SDKs, the stream, the warehouse, the feature store and the experiment tools, and Data Workers works across all of them.

In this space specifically. Recommendation engines and personalization services rank items; Data Workers keeps the events, catalogs and features under them correct. For the sibling teams that share your event data, see clickstream and product analytics, SaaS usage and billing data and geospatial and marketplace data.

The case for your CTO

What this replaces. Days of tracing a click-through drop to a changed event, hand-rebuilt training sets after catalog changes and offline-online comparisons done one feature at a time.

What your team gets back. A personalization data team that ships new signals and candidate generators instead of explaining why the ranker got worse.

Integration effort. Connecting is configuration, not a migration. The agents attach over MCP to the stream registry, warehouse, orchestrator, dbt project and feature store you already run, under the roles you already govern, inside your own infrastructure. Your engineers can read every tool the agents call in the open-source core.

Risk and control. Autonomy is set per domain. Everything starts at L1, observing only; reversible work moves up only when the receipts justify it; production training sets and consent data stay at L2 with a named approver; model promotion stays with people; and one org-wide stop halts all autonomous work.

Why now. Personalization keeps adding surfaces, and each one adds events, features and experiments that can break silently. Agentic features such as AI shopping and content assistants also read the same catalogs and signals, which raises the cost of a wrong one.

The first win. Instrumentation drift and training-serving parity for one ranking surface, at L2. The pilot below lays out the scope, what you see each week and the cost.

The climb: from assisted to autonomous to agentic

Our thesis is that the data platform climbs the way analytics did, in stages, and that the hard part of data work was never knowing what to do but doing it. For personalization and recommendations data teams, the climb looks like this:

• •Assisted. Agents work alongside your team. They read the whole stack, show what needs doing and propose every fix with a receipt; training sets, consent data and model promotion always go to a named approver.
• •Autonomous, under your governance. Routine work runs on its own: event schema changes are mapped into staging the day they happen, duplicate and late events are repaired in the cleaned layer, and batch and stream features are tested for parity every day. Your team governs the agents instead of doing the upkeep.
• •The agentic enterprise. A personalization data team that ships new signals and candidate generators instead of explaining why the ranker got worse. People set direction, and the platform carries it out.

You choose the altitude, every step is visible and reversible, and the pilot below is the first rung.

The pilot: how personalization and recommendations data teams start

The pilot is how your team proves the value on its own data before committing.

• •What we connect. Your event streams, warehouse, dbt project, orchestrator, feature store and experiment tools, over MCP, inside your infrastructure. Nothing migrates.
• •Scope. Instrumentation drift and training-serving parity for one ranking surface, at L2.
• •First week. Agents run at L1 across the scoped surface. They map every event, feature and lineage path from client to model and show the fixes they would have made, with receipts, so your team sees the value before anything changes.
• •By week two. Event schema mapping and duplicate and late event repair start running on reversible work in staging. Training sets and consent data stay at L2 with a named approver.
• •At day 30. We review the impact metrics against your baseline:

• •Broken event traced to its release: from about two days to minutes
• •Training set rebuilt after a catalog change: from about a day to under an hour
• •Training and serving skew traced to its cause: from about three days to under an hour
• •the share of catalog and docs upkeep closed autonomously, targeting 88% as the deployment matures

• •What you decide. Whether to expand to the next ranking surface. The pilot fee is credited in full against the first year.
• •Cost. Start with a pilot ($7,500 one-time, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.

The pilot is the start of a longer curve:

By month 12, more than 40% of agent actions run fully autonomously and manual work falls below 10%, while production training sets, consent data and model promotion stay with a named approver.

FAQ

Does it replace our feature store? No. Feast, Tecton or your own store keeps serving features. Data Workers checks the data under it and the parity between batch and online values.

Will agents change the training set of our production ranker? Training sets for production rankers are L2: proposed with a diff and approved by the ranking lead.

Can it help with experiment readouts? Yes. It checks assignment, exposure and metric definitions before the readout and records which data each arm read. The call to ship stays with people.

How does it handle consent and deletion requests? It finds personal data across logs, features and training snapshots and checks that your pipelines honour your consent platform. Changes to consent-related data go to the privacy lead.

Where do the agents run? In your infrastructure, with your credentials and model key. The hosted Conductor sees workflow metadata only.

Do we have to migrate anything? No. Your streams, warehouse, dbt project, orchestrator, feature store and experiment tools stay where they are.

Sources

• •Google, Rules of Machine Learning (training-serving skew): https://developers.google.com/machine-learning/guides/rules-of-ml
• •Feast, point-in-time joins: https://docs.feast.dev/getting-started/concepts/point-in-time-joins
• •Data Workers pricing: https://dataworkers.io/pricing/
• •Snowflake, managed MCP server and Cortex Agents GA release note (Nov 4, 2025): https://docs.snowflake.com/en/release-notes/2025/other/2025-11-04-cortex-agents-mcp
• •Databricks, Genie One launch (Jun 16, 2026): https://www.databricks.com/company/newsroom/press-releases/databricks-launches-genie-one-all-new-agentic-coworker-every-team