PII Audit: By Hand vs With an AI Agent

A manual PII audit means enumerating every column, sampling values, matching them against patterns, checking who can read them, and writing the findings into a report by hand. An agent running the dw-pii-audit skill does the scan and drafts a remediation-ready report from the same datasets, and a person still approves every change. The stakes are set by regulation: GDPR Article 83 allows administrative fines of up to 4% of worldwide annual turnover or 20 million euros, whichever is higher, for the most serious infringements.

Key Takeaways

• •A manual PII audit is a chain of handoffs: schema export, value sampling, pattern matching, access review, and a written report. Each handoff is where findings get lost or go stale.
• •The dw-pii-audit skill, run by the Governance Agent, scans datasets for PII exposure and produces a remediation-ready audit report. That is the whole scope we claim for it.
• •An agent replaces the repetitive scanning and report assembly. It does not replace the decision to mask, drop, or re-permission a column. A person approves those.
• •Evaluate any PII audit agent on recall against a labeled sample, false-positive rate, read-only behavior by default, and whether every finding links to evidence you can check.
• •Detection libraries such as Microsoft Presidio are useful building blocks, but a library is not an audit. The audit is the repeatable workflow and report around it.

The job and who does it today

A PII audit answers one question for a regulator, a customer security review, or your own risk register: where does personal data live in our data stack, and who or what can reach it? The question sounds simple. In practice it spans the warehouse, the lake, the pipelines that feed them, and the notebooks that copy tables around. Personal data rarely stays in the table where it was first declared.

Today the work usually lands on a mix of people. A data engineer exports schemas and writes sampling queries. A security or privacy analyst reviews the matches and decides what counts as personal data under the policies the company has adopted. A governance or compliance owner assembles the report and tracks remediation tickets. In smaller teams, one person plays all three roles on top of their normal work.

The audit tends to happen on a schedule driven by an external event: an annual certification, a customer questionnaire, a new regulation, or an incident. That timing matters. An audit done once a year describes the warehouse as it was at one moment, while new tables and columns appear every week. Much of the pain comes from this mismatch between a periodic audit and a continuously changing schema.

The manual workflow, step by step

Step one is inventory. Someone queries the information schema, or exports from the catalog, to list every database, schema, table, and column. In a warehouse like Snowflake this is a few queries. Across several warehouses, a lake, and operational replicas, it is a spreadsheet that someone has to keep merging and deduplicating by hand.

Step two is classification by name. Engineers filter column names for obvious hints such as email, phone, ssn, dob, or address. This is fast and catches the well-labeled columns. It misses columns named c14, free-text notes fields, JSON blobs, and any table where a developer chose an unhelpful name. Step three is classification by content: sampling rows and running regular expressions or a detection library such as Microsoft Presidio over the values to find emails, card numbers, and national identifiers that the names did not reveal.

Step four is the access review. For each flagged column, someone lists the roles, users, and service accounts that can read it, then compares that list against who should. Snowflake, for example, documents a 365-day retention window for the ACCESS_HISTORY view, so a year of read activity is queryable, but turning raw history into a clear answer for each flagged column is still manual query work. Step five is the report: findings, severity, owners, and recommended remediation, written into a document and then into tickets.

Where the manual workflow breaks

The first break is coverage. A manual audit samples. Sampling a few thousand rows from a billion-row table can miss a rare PII pattern that only appears in a slice of the data, such as a support-notes column where customers occasionally paste a phone number. Teams accept this because scanning everything by hand is not feasible, and the risk is rarely written down.

The second break is handoffs. The engineer who sampled the data, the analyst who judged the matches, and the owner who wrote the report each hold part of the context. Decisions such as why a column was judged not to be PII live in chat threads and are gone by the next audit. When the same audit runs again six months later, a new person starts from scratch and often reaches different conclusions about the same column.

The third break is staleness. The report is accurate on the day it is signed and drifts from that moment. A new table with raw customer exports can appear the following week and sit unreviewed until the next cycle. The fourth break is that the report is not remediation-ready. It lists findings, but the person who has to fix them still has to work out which pipeline writes the column, what depends on it, and what a safe change looks like.

What an agent does instead

The Governance Agent packages this job as the dw-pii-audit skill. The skill description is short and we keep our claims to it: it scans datasets for PII exposure and produces a remediation-ready audit report. The agent runs as an MCP server, so an engineer invokes it from Claude Code, Cursor, or any MCP-compatible client rather than opening a separate tool. The Model Context Protocol specification defines how clients discover and call the tools it exposes.

In practice that replaces steps two, three, and five of the manual workflow with one repeatable run. The scan covers the datasets you point it at, and the output is a report organized around what needs fixing, not only what was found. Because the run is a command rather than a project, it can be repeated after a schema change instead of once a year. The OSS edition is read-only by default, so a scan observes your data and does not alter it.

Questions that used to take a custom query become prompts, such as who has access to PII columns in analytics. The agent can chain with other agents in the swarm when the audit needs context beyond the scan, such as lineage to see what depends on a flagged column. We do not claim the agent makes legal judgments. It produces evidence and a structured report, and the people accountable for the decision read it.

Manual vs agent: side-by-side

The table below compares the two approaches on the dimensions that matter during an audit. We describe effort qualitatively, because the time a manual audit takes depends on the size of the estate and the maturity of the catalog, and we do not have a public benchmark to cite for it.

Honest caveat: the manual approach wins where context lives only in people's heads. If a column is ambiguous and only the product owner knows what it holds, no scan resolves that. The agent shifts the effort from finding candidates to judging them, which is where human attention is better spent.

What still needs a human approval

The report is the start of remediation, and the actions in it change production data or access. Those need approval. Masking a column, dropping it, changing a role grant, or adjusting a retention period can break dashboards and downstream jobs, and the right choice depends on business context an agent does not hold. In the OSS edition these writes are not performed at all. In Pro, writes go through approval workflows and are recorded in audit trails.

A person also owns the classification policy. What counts as sensitive, which columns are acceptable to hold in clear text, and which jurisdictions apply are decisions for privacy and legal owners. The agent applies the policy you give it and surfaces candidates. It does not decide that a column is exempt from your obligations, and we would be wary of any tool that claimed to.

Finally, a person signs the report. An auditor or regulator asks who attested to the findings, and that has to be an accountable human. A sensible split is that the agent runs the scan and drafts the report, an analyst reviews flagged and borderline findings, and the governance owner approves the remediation list before any ticket or change is created.

How to evaluate an agent for this job

Start with a labeled sample. Pick a few schemas where your team already knows where the PII is, including awkward cases such as free-text fields, JSON blobs, and cryptically named columns. Run the agent and measure how many known PII columns it found (recall) and how many of its flags were wrong (false positives). Both numbers matter. Low recall leaves exposure unreported, and high false positives bury reviewers in noise.

Next, check the safety posture. Confirm the agent is read-only by default, that any write requires explicit approval, and that sample values do not leave your infrastructure. Our posture is that data stays in the customer's environment, and you should hold any vendor to a clear statement on that. Ask what is logged, whether the audit trail is tamper-evident, and whether PII in the report itself is redacted.

Then check the output. Every finding should point to evidence you can verify: the table, the column, and the reason it was flagged. Run the agent twice on the same data and confirm the results are stable. Finally, ask how the report maps to remediation: does it tell the engineer what to change, and does it fit your ticketing and approval process? An agent that finds PII but produces a report nobody can act on has only automated the easy half.

Frequently Asked Questions

Can an AI agent replace a manual PII audit entirely?

No. The agent replaces the repetitive scanning and report assembly, not the accountable decisions. A person still sets the classification policy, reviews borderline findings, approves remediation, and signs the report. What changes is where the effort goes: less time finding candidates and writing documents, more time judging the ones that matter.

What does the dw-pii-audit skill actually do?

It scans datasets for PII exposure and produces a remediation-ready audit report, and it is run by the Governance Agent. We limit our claims to that description. It is invoked through MCP from a client such as Claude Code or Cursor, and in the OSS edition it is read-only by default, so running a scan does not change your data.

Is a detection library like Presidio enough on its own?

A library detects entities in text and is a good component. An audit also needs inventory, coverage decisions, access review, evidence, and a report an owner can act on. Presidio gives you the detection step. The workflow around it is the part teams end up building by hand, and that is the part an agent is meant to take over.

How often should a PII audit run?

Often enough that the report does not drift far from the warehouse. Regulations and certifications set minimums for some organizations, and your own policy sets the rest. Because an agent run is a command rather than a project, teams can re-run it after significant schema changes instead of waiting for the next annual cycle.