Chinese Academy of Sciences' BiNSGPS Hits 95.7% on Geometry Solving With Self-Correcting AI
BiNSGPS adds a feedback loop between a vision-language model and a symbolic solver, hitting 95.7% on MathVista's geometry benchmark
Takeaways
- BiNSGPS introduces a bidirectional feedback loop between a multimodal LLM and a symbolic solver for geometry problem solving, replacing brittle one-way pipelines.
- Achieves 95.7% accuracy on MathVista's GPS benchmark, up from 82.7% for the previous best open-source model.
- When the symbolic solver hits a contradiction, it sends a structured error report back to the MLLM Adviser, which either corrects the formal representation or proposes auxiliary geometric constructions.
- The feedback loop runs up to 3 iterations; unresolved problems drop from 170 to 34 across iterations, showing the mechanism actively resolves failures.
- Evaluated on Geometry3K, PGPS9K, and MathVista-GPS benchmarks with consistent gains over unidirectional baselines.
- No code or model weights released yet; current scope is limited to plane geometry and may struggle with corner cases like tangential circles or solid geometry.
Getting AI to solve geometry problems reliably has been a stubborn open challenge. The difficulty isn't just computation -- it's that the task demands both visual understanding (reading a diagram) and rigorous logical deduction (applying theorems step by step). A new paper from researchers at the Chinese Academy of Sciences introduces BiNSGPS, a framework that rethinks how neural models and symbolic solvers collaborate -- and the results push the state of the art on MathVista's geometry problem solving (GPS) benchmark to 95.7%.
Two camps, two failure modes
Geometry problem solving requires exceptional multimodal comprehension and rigorous mathematical reasoning, and existing approaches typically fall into two categories -- neural-based and symbolic-based methods -- both of which exhibit limitations in reliability and interpretability.
- Symbolic solvers work by translating a problem into formal language and then searching through a theorem database to derive a solution. They feature explicit and clear steps in the solution and reasoning process. But they often rely heavily on manually crafted rules and formally annotated data, which limits their scalability and adaptability.
- Neural solvers (including large multimodal models) are flexible and can handle diverse inputs, but applying MLLMs to geometry problem solving remains challenging due to lack of accurate step-by-step solution data and severe hallucinations during reasoning.
Recent advances in large models have enabled notable breakthroughs, particularly for SAT-level problems, yet the field remains fragmented across methodologies, benchmarks, and evaluation frameworks. The hybrid approaches that have emerged -- neuro-symbolic pipelines -- mostly feed neural outputs into a solver and stop there. If the neural model misreads the diagram or writes a bad formal representation, the solver fails with no way to recover.