Why Text-to-SQL Without a Semantic Layer Is Dangerous

Text-to-SQL is the most overpromised feature in data tooling today. Every vendor demos it. "Ask your data anything in plain English!" The demo always works. Production never does. Google's benchmarks show frontier models begin to struggle when applied to real-world enterprise database schemas. The Beaver benchmark for Text-to-SQL shows accuracy drops dramatically on complex schemas. The reason: the models do not understand what the data means.

Why Raw-Table Querying Fails

The model does not know which "revenue" column to use. There are three tables with revenue data — one is gross, one is net, one is recognized. The model picks one. It does not know that orders should be filtered by is_deleted = false. It does not know that your fiscal year starts in February. It does not know that the customer_id in the events table is actually device_id for anonymous users.

Every wrong assumption compounds into confidently wrong answers. The model does not hedge — it returns a number. A specific, precise, well-formatted, completely wrong number. And the person who asked the question has no way to know it is wrong.

The 66% Gap

Google's benchmarks show 66% accuracy improvement when queries are grounded in a semantic layer. This is not a marginal improvement — it is the difference between usable and unusable. A tool that is wrong a third of the time is not a tool. It is a liability.

The semantic layer tells the model what business terms mean, how metrics are calculated, and what filters are always required. "Revenue" is not ambiguous when there is a governed definition that says: net revenue, USD, post-refund, recognized at booking date, excluding internal test accounts. The model does not need to guess. It follows the definition.

What the Data Science and Insights Agent Does

The Data Science and Insights Agent is text-to-SQL done right:

• •Semantic layer grounding. Every query is generated against your semantic layer (dbt Semantic Layer, Looker LookML, Cube.dev, AtScale) — not raw tables. The agent uses governed metric definitions, not guesses.
• •Metric disambiguation. When someone asks about "revenue," the agent does not pick a table at random. It asks: "Did you mean net revenue ($4M last quarter), gross revenue ($5M), or recognized revenue ($4M)?" and shows the governed definition for each.
• •Auto-visualization. Results are returned with appropriate charts — time series for trends, bar charts for comparisons, tables for detail. Not just raw query results.
• •Cited answers. Every answer traces back to specific tables, columns, and rows. The user can verify. Trust requires transparency.
• •Scheduled reports. Set up recurring queries that run on a schedule and deliver results to Slack, email, or dashboards.
• •Multi-hop reasoning. Answers questions that require joining across multiple tables and applying multiple filters. "What is the average order value for customers who signed up in Q1 and made at least 3 purchases?" requires joins, filters, and aggregations across several tables.

What This Is Not

This is not a chatbot on top of your warehouse. It is a semantic-aware query agent that validates every query against governed definitions before execution. It does not generate SQL and hope for the best. It generates SQL that is constrained by your organization's definitions of what the data means.

The difference matters when the CEO reads the output. A chatbot gives you an answer. This agent gives you an answer you can trust — because it is grounded in the same definitions your finance team uses.

Key Metrics

• •Time-to-insight: hours/days to seconds. Business users get answers without waiting for a data analyst to write a query, review it, and send the results.
• •Self-service analytics adoption increases 5x. When answers are trustworthy, people actually use the tool. When they are not, the tool gets abandoned within weeks.

Text-to-SQL without a semantic layer is dangerous. Text-to-SQL with a semantic layer is transformative. The semantic layer is not optional — it is the entire difference between a demo and a product.