Tencent's ProSPy Solves Real Enterprise Databases Where Every Other AI Agent Fails
ProSPy combines SQL data retrieval with Python analysis in a four-stage agentic pipeline, hitting 60% accuracy on the notoriously hard Spider 2.0 enterprise benchmark
- ProSPy is a four-stage agentic framework from Zhejiang University and Tencent TEG that combines SQL retrieval with Python analysis for enterprise Text-to-SQL.
- It achieves 60.15% and 60.51% execution accuracy on Spider 2.0-Lite and Spider 2.0-Snow with Claude Opus 4.5, beating all prior methods by ~24 points without majority voting.
- A key innovation is a dialect-agnostic DSL that abstracts SQL generation away from specific database engines, making the same pipeline work across Snowflake, BigQuery, and SQLite.
- Automatic data profiling replaces expensive iterative database exploration: each database is profiled once in parallel and reused across all questions.
- Progressive schema pruning achieves 86% precision (vs. 30-54% for competitors), keeping context compact and reducing noise for downstream reasoning.
- No code is publicly released, but full prompts are in the paper appendix; the framework works with DeepSeek V3.2, GLM-5, and Claude Opus 4.5.
Getting an LLM to write SQL for a toy database is a solved problem. Getting it to answer a complex business question against a real enterprise database -- one with thousands of columns, ambiguous naming conventions, Snowflake-specific syntax, and no reliable documentation -- is a completely different challenge. A new paper from Zhejiang University and Tencent TEG tackles exactly that with ProSPy, a four-stage agentic framework that pairs SQL-based data retrieval with Python-based analysis to handle the messiness of production databases.
Why enterprise Text-to-SQL is still broken
The standard Text-to-SQL benchmarks -- Spider 1.0 and BIRD -- have been largely conquered. Models routinely score above 85% on Spider 1.0. But those benchmarks use clean, small schemas with well-documented columns and a single SQL dialect. Real enterprise databases look nothing like that.
Spider 2.0, introduced at ICLR 2025, was designed to close that gap. It comprises 632 real-world text-to-SQL workflow problems derived from enterprise-level database use cases, with databases often containing over 1,000 columns stored in cloud systems such as BigQuery and Snowflake. The difficulty jump is staggering: the best agent framework based on o1-preview solves only 21.3% of tasks, compared with 91.2% on Spider 1.0 and 73.0% on BIRD.
The core problems are structural. Solving Spider 2.0 problems requires models to interact with complex SQL workflow environments, process extremely long contexts, perform intricate reasoning, and generate multiple SQL queries with diverse operations, often exceeding 100 lines. On top of that, metadata is often wrong or missing, and a single monolithic SQL query simply cannot express many real analytical questions.
The four-stage pipeline
ProSPy's central insight is that SQL and Python have complementary strengths. SQL is efficient at querying large databases and doing lightweight aggregation. Python is flexible enough to handle multi-step analysis, ranking logic, and statistical computation -- but it can't efficiently scan a billion-row table. ProSPy uses each tool for what it's good at, connecting them through a structured pipeline.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
