Agentic Semantic Search: By Hand vs With an AI Agent
What the manual data-asset search loop looks like, what an agent does instead, and what still needs a human
Agentic semantic search is a loop in which an agent plans a query, searches your data assets, reads what comes back, re-queries when the results are weak, and stops only when it can name the right assets with evidence for each. Done by hand, the same loop is a mix of catalog keyword search, warehouse metadata queries and questions to colleagues. In this guide we compare the two so you can decide which parts of discovery to hand to an agent.
Key Takeaways
- •Manual data discovery is an iterative loop (guess a name, search, inspect, doubt, search again), and the agentic version automates that loop rather than a single lookup.
- •Our dw-agentic-semantic-search skill, run by the Catalog Agent, plans, searches, re-queries and converges on the right assets with provenance.
- •Provenance is the deciding feature: an answer that cannot say where it came from is a guess, whether a person or an agent gives it.
- •Agents should find and recommend assets; a human still approves anything that changes access, definitions or production usage.
- •Evaluate an agent for this job on whether it converges, whether it shows its sources, and whether it admits when no good asset exists.
The job: finding the right data asset, and who does it today
Almost every data request starts the same way. An analyst, an engineer, or increasingly another agent needs a table, column, dashboard or metric, and does not know exactly what it is called or whether three near-duplicates exist. Today the person asking is usually also the person searching. Senior engineers act as a human index, answering "which orders table is the real one?" from memory because the catalog does not settle it.
The cost of getting this wrong is quiet. A join on a deprecated table does not throw an error. It returns plausible numbers that disagree with finance. That is why discovery is a correctness problem and not just a convenience problem, and why a bad pick propagates into reports, models and downstream agents without anyone noticing.
The job has two halves. The first is retrieval: surface candidates that could match the intent. The second is judgment: decide which candidate is trustworthy, current and appropriate for this question. Keyword search handles a thin slice of retrieval. Judgment has historically been entirely human, and it is the half that makes the manual loop slow.
The manual workflow, step by step
A typical manual search runs in five handoffs. First, the person types a guess into the catalog search box. Second, they open two or three hits and read column names and descriptions, which are often empty or stale. Third, they run exploratory queries in the warehouse to check row counts, freshness and sample values. Fourth, they look at lineage, if it exists, to see what feeds the candidate. Fifth, they ask a colleague in chat to confirm.
Usage history helps when you can reach it. For example, Snowflake documents that its ACCESS_HISTORY account usage view retains records for 365 days, which is enough to see which tables people actually read. But that means writing and interpreting a query against a system view, which most requesters do not do. The evidence exists, yet it sits behind another manual step.
Where a semantic layer is defined, it narrows the problem. The dbt semantic models documentation describes entities, dimensions and measures declared in code, so a metric has one definition. That helps for governed metrics. It does not help when the question is about a raw table, a column nobody documented, or an asset outside the modeled set.
Where the manual loop breaks
The first failure is vocabulary. People search with business words ("churned customers") and assets carry engineering names ("fct_subscription_end"). Keyword matching cannot bridge that gap, so the searcher tries synonyms one at a time. Each attempt costs a few minutes, and the loop ends when the person gets tired, not when the answer is confirmed.
The second failure is lost reasoning. Even when someone finds the right table, the evidence for why it is right (freshness check, owner confirmation, lineage path) lives in their head or a chat thread. The next person repeats the whole loop. Teams that write the answer back into catalog descriptions help, but that depends on discipline that rarely survives a busy quarter.
The third failure is tool fragmentation. The catalog, the warehouse, the lineage view and the chat history are separate systems, and the human is the integration layer. Nothing in that chain checks whether the candidate is deprecated, recently changed or restricted. Those checks are possible, but only if the searcher remembers to run them.
What an agent does instead
The Catalog Agent runs the dw-agentic-semantic-search skill, which we describe as iterative agentic search over data assets that plans, searches, re-queries, and converges on the right assets with provenance. The important word is iterative. A single embedding lookup returns the nearest neighbors and stops. The skill treats the first result set as a draft and keeps going.
In practice that means the agent turns a plain-language question into a search plan, runs the searches, inspects what came back, and decides whether the results answer the question. If they do not, it rewrites the query and searches again. When the candidates stabilize, it returns the assets together with provenance, meaning where each match came from, so a reader can check the reasoning instead of trusting it.
Because every Data Workers agent is an MCP server, you invoke this from the client your engineers already use. The Model Context Protocol specification defines how a client discovers and calls tools exposed by a server, so the same search can be called from a terminal assistant, an editor or another agent without a new interface to learn.
What still needs a human
The agent finds and explains. It should not decide on its own that a table is now the canonical source for a metric, because that is a governance decision with consequences for every consumer. We keep that approval with a person: an owner or steward confirms that the recommended asset becomes the documented answer.
The same applies to access. Discovery surfaces what exists, and it can reveal that a sensitive column exists. Whether the requester should be allowed to read it is a policy question for a human approver. Our open-source edition is read-only by default, so searching is safe to try before any write path is enabled.
Humans also own the ambiguous cases. When two assets are both defensible, or the business definition is contested, the right output is a short list with evidence and a question for the owner. An agent that picks one silently is worse than a slow person, because it hides the disagreement.
Manual vs agent: side by side
| Stage | By hand | With the agent |
|---|---|---|
| Interpret the question | Searcher guesses names and synonyms | Agent plans the search from the plain-language question |
| Retrieve candidates | One keyword query at a time in the catalog | Agent searches, then re-queries when results are weak |
| Check candidates | Open each hit, run exploratory SQL, check lineage | Agent inspects results and converges on the likely assets |
| Record the reasoning | Lives in chat threads and memory | Returned as provenance with the answer |
| Final decision | Searcher decides alone | Human owner approves anything that changes definitions or access |
| Effort | Repeated by each new requester | Qualitatively lower; the loop runs without the requester in it |
We deliberately do not put minute or dollar figures in this table. We have no public benchmark we can attribute for discovery time, and any number we invented would be worthless to you. Measure your own baseline: time a handful of real discovery requests today, then compare.
How to evaluate an agent for this job
Start with a test set drawn from your own history: ten to twenty questions where a person already found the right asset. Run the agent on them and score whether it lands on the same asset. Include a few questions where the correct answer is "there is no good asset", because an agent that always returns something is dangerous.
Next, inspect provenance. For each answer, can you trace every recommended asset back to the search that produced it and the evidence it used? If the agent gives confident answers with no trail, you cannot audit it, and you cannot improve it. This is the single most useful filter we know of.
Finally, check where it runs and what it can touch. Look for read-only defaults, whether data stays in your infrastructure, and whether it federates across the catalogs you already run. Open projects such as OpenMetadata on GitHub show what a metadata store exposes, and a good agent should work with that metadata instead of asking you to rebuild it.
Frequently Asked Questions
How is agentic semantic search different from a normal semantic search?
A normal semantic search embeds the query, returns the closest matches, and stops. Agentic search wraps that in a loop: plan, search, evaluate, re-query, converge. The difference shows up on vague questions, where the first result set is rarely right and a second or third query based on what came back is what finds the correct asset.
Does the agent replace our data catalog?
No. The agent searches over assets and metadata that already exist, and the Catalog Agent) federates across the catalogs you run. Your catalog stays the system of record. The agent reduces the manual searching and cross-checking on top of it, and returns provenance so the result can feed back into catalog documentation.
Can an agent be trusted to pick the canonical table?
It can recommend, with evidence. Declaring a table canonical is a governance decision, so we keep a human owner in that step. Treat the agent's output as a well-sourced proposal that a steward accepts or rejects, and treat a silent single answer with no sources as a red flag.