Proof-Refactor Turns Ugly AI Math Proofs Into Human-Readable Code
Proof-Refactor is a four-phase agentic system that turns monolithic LLM-generated Lean proofs into readable, modular, library-quality artifacts
Takeaways
- The problem: LLM-generated Lean proofs compile but are monolithic, unreadable, and hard to reuse -- a side effect of training purely on correctness signals.
- The solution: Proof-Refactor is a four-phase agentic framework that extracts sub-proofs, designs named helper lemmas, proves them independently, and rewires the original proof.
- Key results: Outperforms a Claude Code refactoring baseline on rubric-based scores across PutnamBench and Putnam2025, with the biggest gains in signature quality and human readability.
- Novel approach: Uses a custom
lean_extracttactic to surgically lift tactic blocks into standalone theorems, grounded in the Lean compiler at every step. - Limitation: A post-processing tool only -- requires an already-correct Lean proof as input and depends on external models for sub-goal proving.
- Open source: Code is available at github.com/pelicanhere/proof-refactor.
LLMs have gotten surprisingly good at generating formal proofs in proof assistants like Lean. But there is a catch: the proofs they produce tend to compile and nothing more. They are tangled, monolithic scripts that a human mathematician would never write, let alone want to maintain or reuse. A new paper from researchers at Southern University of Science and Technology and Peking University proposes Proof-Refactor, an agentic framework that takes those working-but-ugly proofs and restructures them into something closer to what you would find in a mature math library.
The compile-first trap
Formal theorem proving with proof assistants such as Lean produces machine-verifiable mathematical proofs. This is useful for verified mathematics and for training LLMs, since proof checking provides a precise reward signal for reinforcement learning. The problem is that this same reward signal creates a perverse incentive.
Proof-assistant verification induces a narrow objective: a generated proof is rewarded primarily for compiling. This binary feedback ensures correctness, but it does not by itself encourage readability, modularity, maintainability, or library integration. The result is what the authors call the "compile-first" trap: the compile-first objective implicit in most proof-generation pipelines encourages monolithic or ad hoc proof scripts rather than library-quality artifacts.
In practice, LLMs trained with reinforcement learning often produce Lean proofs that are correct but notoriously long and hard to read. Prior work tried to fix this, but ran into a measurement problem: the most tractable and experimentally validated objectives are largely length-based, while higher-level qualities such as readability, modularity, maintainability, and reusability are difficult to reduce to reliable automatic metrics.
A process-guided answer
Instead of chasing a single proxy metric, the Proof-Refactor team took a different angle: they take a process-guided approach inspired by human proof-refactoring workflows. The idea is that the right way to improve a proof is not to optimize a score, but to follow the same steps a skilled mathematician would take when cleaning up someone else's draft.