Skip to main content
Back

Fidelity's Text2Model Turns Plain English Into Solver Code but Struggles at 38%

A new benchmark and copilot suite tests whether LLMs can turn plain-English problem descriptions into working optimization models, with mixed results.

  • Text2Zinc offers 1,775 natural-language optimization and satisfaction problems in a unified schema, with 110 hand-verified instances.
  • Text2Model benchmarks eight LLM strategies from zero-shot to agentic pipelines against these problems using MiniZinc as the solver-agnostic target.
  • Chain-of-Thought with grammar validation was the strongest strategy, solving 956 of 1,318 instances with GPT-5.2.
  • 27 percent of problems remained unsolved by any tested strategy, showing text-to-model is still far from push-button.
  • Solver-specific fine-tuning (OptiMind on Gurobi) hurt general modeling ability compared to prompting frontier models.
  • Everything is open source on the HuggingFace leaderboard and dataset, accepting new submissions.

Turning a business problem written in plain English into a working constraint model has always been a job for specialists. A new research effort called Text2Model and its accompanying Text2Zinc dataset put large language models to the test on exactly that task, and the results suggest we are not quite at the push-button stage yet.

The work comes from researchers at Fidelity Investments' AI Center of Excellence and Brown University. They formalized the text-to-model translation problem, proposed several Text2Model copilots with an online leaderboard, and contributed the Text2Zinc dataset to enable benchmarking across both optimization and satisfaction problems in a solver-agnostic manner. Everything, including an interactive editor with a built-in AI assistant, is open source.

The modeling bottleneck nobody wants to talk about

Optimization solvers have gotten dramatically faster over the past few decades, but the workflow around them has barely changed. Someone with deep expertise still has to translate a messy real-world problem into a formal specification of variables, domains, and constraints before a solver can touch it. Domain experts who understand their problem often lack the specialized knowledge required for formal modeling, and the resulting dependency on modelers creates operational bottlenecks and can introduce misinterpretation of domain-specific requirements during translation.