Text-to-SQL in 2026: The Benchmarks Got Solved. Your Database Didn't.

Picture this: a sales lead types “How many active customers did we have last quarter?” into the company’s shiny new AI data assistant.

The model writes clean, valid SQL in under a second:

SELECT COUNT(*)
FROM customers
WHERE status = 'active'
  AND created_at >= '2026-07-01';

It runs. It returns 0.

Nothing is wrong with the SQL. Everything is wrong with the context. In this database, status is stored as 'A', “active” means placed an order in the last 90 days, and the real answer lives in a table called cust_ord_agg_v2 that nobody has documented since 2021.

That gap, between writing SQL and writing the right SQL for your database, is the whole story of text-to-SQL in 2026.

The benchmarks moved faster than anyone expected

Two benchmarks matter here.

BIRD uses real databases with dirty values: 95 databases, 33.4 GB, 37 domains. Human data engineers score 92.96%. As of October 2026, the top system on the leaderboard, AT&T’s AskData, scores 83.40% on the hidden test set.

Spider 2.0 is the harder one. It’s built from real enterprise warehouses on BigQuery and Snowflake, often with more than 1,000 columns per database. When it launched in late 2024, the best agent (built on o1-preview) solved just 21.3% of tasks, against 91.2% on the original Spider. That gap was the headline: models that looked solved on toy schemas fell apart on real ones.

Two years later, the leaderboard looks very different:

Spider 2.0 setting	What it tests	Top score (Nov 2026)
Snow	547 text-to-SQL tasks on Snowflake, with prepared metadata and docs	96.70%
Lite	547 tasks across BigQuery, Snowflake and SQLite	76.23%
DBT	68 project-level data transformation tasks	65.6%

So is text-to-SQL solved? Not quite. Read the leaderboard closely and two things stand out:

Almost every top entry is a heavily engineered, mostly closed agent system from a company like Tencent, Ant Group, ByteDance or AT&T. Not a model on its own.
The Snow setting ships with well-prepared metadata and documentation. Your production warehouse probably doesn’t.

The jump from 21% to 96% didn’t come from a smarter model writing better SQL. It came from everything built around the model.

What the top systems actually do

Read through the papers behind these numbers and the same five patterns keep showing up.

Five patterns top text-to-SQL systems build around the model, combined from several papers

1. Find the right tables before writing anything

This step is called schema linking: pick the handful of relevant tables and columns out of thousands before generating SQL. CHESS gives it a dedicated “Schema Selector” agent. RSL-SQL links in both directions, from the question to the schema and back. LinkAlign goes a step further and first picks the right database out of a pool of them. EviLink treats it as an uncertainty problem: when a question could be answered several ways, it fetches more evidence only where the paths disagree.

2. Let the agent explore instead of guessing

Instead of one big retrieval step, AutoLink gives the LLM tools and lets it explore the schema iteratively, expanding what it looks at as it learns more. It reaches 97.4% schema recall on BIRD-Dev and 91.2% on Spider 2.0-Lite, without ever loading the full schema into the prompt.

3. Look at the actual data

Column names lie. Values don’t. ProSPy starts by profiling the database: sampling values, formats and distributions to pull out evidence that metadata doesn’t contain. This is the step that would have caught 'A' instead of 'active' in the opening example. ProSPy reaches about 60% on both Spider 2.0-Lite and Snow with Claude 4.5 Opus, without majority voting.

4. Check the SQL before trusting it

CHESS generates candidates, refines them iteratively, then runs them past a “Unit Tester” agent that checks each query against LLM-written natural-language tests. The general pattern is the same everywhere: execute, look at the error or the suspiciously empty result, and try again.

5. Generate several answers and pick one

LLMs are sensitive to how you prompt them. MCS-SQL turns that weakness into a strategy: generate candidates from multiple prompts, then pick the best one with a multiple-choice selection step. The BIRD leaderboard now labels each entry by how many candidates it votes over, from none up to “Scale” (more than 32).

Where schema linking isn’t worth it

Before you build an elaborate retrieval pipeline, two results should give you pause.

“The Death of Schema Linking?” found that strong models handle irrelevant tables well, and that filtering the schema often removes columns the query needed. Their pipeline skips schema linking entirely whenever the schema fits in the context window, and it took first place on BIRD at the time, with 71.83%.

Spider 2.0-AIFunc, a newer benchmark of AI-native SQL on Snowflake, found that “a minimal agent setup consistently matches or outperforms more elaborate alternatives.”

Rule of thumb: if the schema fits in the context window, show the model everything. Reach for schema linking when it doesn’t.

Where the open-source tools stand

The research gives you the techniques. Here’s what you can actually install today.

Vanna popularised the simplest working recipe: store your DDL, docs and known-good question-to-SQL pairs in a vector store, retrieve the closest ones, and prompt the LLM. Vanna 2.0 rebuilt it as an agent with row-level security, audit logs and per-user rate limits. Then the GitHub repo was archived, with the last push on 2 February 2026. No official reason has been given.

WrenAI bets on a semantic layer. You describe what your data means (models, relationships, metrics, enums, approved joins) in its Modeling Definition Language. These are Git-friendly files you review in a pull request. Its Apache DataFusion engine plans every query against those definitions across 22+ sources, and it now ships an MCP server so Claude Code and Cursor can use it directly. One catch: row- and column-level security is part of the paid tier.

DBHub (by Bytebase) is the minimalist option: one MCP server for Postgres, MySQL, MariaDB, SQL Server, Oracle and SQLite, with just two tools by default (execute_sql and search_objects). It adds read-only mode, row limits and query timeouts. There’s no AI inside; your agent does the thinking.

MCP Toolbox for Databases (by Google, formerly “GenAI Toolbox”) covers far more sources: BigQuery, Spanner, AlloyDB, Snowflake, MongoDB, Neo4j and others. It also has a framework for defining your own safe tools, including NL2SQL.

What’s still missing

Line the research up against the tools and two gaps are left.

1. Maintained, open-source governance. The project that shipped free row-level security is archived. The most complete active one charges for it. If you want AI agents near production data, permissions shouldn’t be a premium feature.

2. Numbers you can rerun. The leaderboards are full of closed systems. If you’re choosing an open-source tool, you rarely get an accuracy figure you can reproduce on your own machine. BIRD Mini-Dev, with 500 questions in SQLite, MySQL and PostgreSQL, makes this cheap. Ask for it.

That’s what the next posts in this series will build in public: a small, MCP-native text-to-SQL engine that profiles your data, links the schema only when it has to, checks its own SQL, and publishes its BIRD Mini-Dev score with the code to reproduce it.

The model writes the SQL. Everything around it decides whether the SQL is right.

Sources

Benchmarks

Techniques