Peking University's ProfiliTable Beats Every Rival by 30 Points on Messy Data Cleanup
ProfiliTable is a multi-agent framework that actively explores tabular data before writing code, beating every baseline by 30+ points on a new 18-task benchmark.
Takeaways
- ProfiliTable is a 7-agent framework that actively explores tabular data before writing transformation code, accepted at KDD 2026.
- It beats all baselines by 30+ ATS points on single-step tasks and is the only method to achieve 100% task-wise runnable rate on multi-step tasks.
- The key innovation is a dynamic profiling context -- a shared scratchpad combining live data exploration, retrieved operator templates, and execution feedback that all agents read and update.
- Evaluated on a new benchmark of 18 task types and 129 tasks spanning cleaning, transformation, augmentation, and matching, using GPT-4o and GPT-5.2.
- Ablation shows feedback (the Summarizer loop) is the single most critical component; removing it causes the largest performance drop, especially in multi-step pipelines.
- Source code is open; future work targets fine-tuning smaller models on ProfiliTable's reasoning traces via reinforcement learning.
Ask any data engineer where most of their time goes, and the answer is almost always the same: cleaning and transforming tables before the interesting work can begin. It's tedious, error-prone, and surprisingly hard to automate. LLMs can write pandas code on demand, but they routinely produce scripts that run without errors yet produce semantically wrong outputs -- the kind of silent failures that corrupt downstream models without raising any alarms.
ProfiliTable, a new multi-agent framework from Peking University accepted at KDD 2026, attacks this problem with a deceptively simple insight: before writing any transformation code, an agent should actively explore the data to understand what it actually contains. That shift -- from passive metadata consumption to dynamic, hypothesis-driven profiling -- turns out to be the key ingredient that prior systems were missing.
Why LLMs keep failing at table processing
The failure mode is well-understood in practice but rarely addressed head-on. When a user says "standardize the currency column," an LLM has no idea whether that column contains "USD", "$", "dollars", "yuan", "RMB", or some mixture of all five. Existing tools like CleanAgent perform column-type annotation but do not actively sample or inspect actual cell values during code generation. Rule-based systems like DataGovAgent sample only a few rows per column to get basic properties, but this limited inspection often fails to capture the full range of values needed to interpret ambiguous instructions.
The result is code that is syntactically correct but semantically wrong -- it runs, produces output, and silently corrupts your data. ProfiliTable's authors identify three root causes:
- Ambiguous instructions: Natural language instructions rarely specify the exact values or edge cases present in the data.
- Complex task structures: Multi-step pipelines (e.g., deduplicate, then normalize, then join) require coordinated reasoning that single-shot code generation cannot provide.
- No structured feedback: Without a closed loop that checks whether the output actually satisfies the intent, errors compound silently.
The architecture: seven agents, one shared context
ProfiliTable orchestrates seven specialized agents around a single evolving object called the dynamic profiling context. Think of it as a shared scratchpad that grows richer with every iteration, combining three things: the current data profile, retrieved code operator templates, and a history of past execution feedback. Every agent reads from and writes to this context, keeping the whole pipeline semantically grounded.