IWC-Bench Runs AI-Built Web Apps Before Scoring Them

Coding agents often declare a web app finished even when buttons do nothing, routes return 404 errors, or unwanted login screens block access. Tencent’s Hunyuan team has published the IWC-Bench paper, which evaluates generated applications by running them in a browser and recording what a tester can reach.

IWC-Bench, short for Interactive WebCoderBench, treats evaluation as a software-testing problem. Source analysis and screenshot comparisons can reward features that appear complete but remain unreachable at runtime. The benchmark instead gathers evidence from browser interactions, then separately scores visual quality, usability, and compliance with the original request.

Broken apps can pass static checks

Static web-generation benchmarks usually compare source code or a single screenshot with a reference. Those methods can detect visual similarity or the presence of expected components, but they cannot establish whether navigation, forms, and other interactions work.

Existing interactive benchmarks often give a testing agent a fixed checklist or rubric. A failed check can then reflect an application defect, an incomplete test path, or confusion by the agent. IWC-Bench separates exploration from scoring and uses code coverage, the share of instrumented code executed during testing, to determine whether the explorer has reached enough of the application.

Four stages from click to score

The evaluation pipeline completes four stages before assigning scores:

1. Instrument the application. A tool based on Istanbul adds counters to the generated JavaScript so the evaluator can measure which statements and functions execute. The system supports plain HTML, Vite, Next.js, Create React App, and Astro without project-specific adaptation.
2. Explore with coverage guidance. An LLM agent built on AWorld controls the browser through Playwright MCP. It receives a simplified DOM, the browser’s structured representation of the page, and sees only the changes between steps to conserve context. The explorer cannot access the source code or acceptance criteria. After three interactions without a coverage gain, another LLM analyzes the unexecuted code and supplies natural-language suggestions. Testing stops when the explorer finishes or reaches 100 steps.
3. Compress the observed states. The evaluator removes scripts, styles, timestamps, and other unstable data from captured pages. It then merges duplicate observations into a state-transition graph that records the pages and interactions reached during testing.
4. Score the evidence. Separate judges assign scores from 0 to 100 for visual aesthetics, usability, and alignment with the original requirements.

Aesthetics and usability use a three-round review. An advocate identifies strengths, a critic identifies defects, and a final judge checks both assessments against the captured evidence. The final judging step runs five times; the highest and lowest scores are discarded, and the remaining three are averaged.

Requirement alignment uses a different agent. It retrieves DOM snapshots, interaction traces, and screenshots as needed, then returns a binary result for each acceptance criterion. Every verdict must cite runtime evidence.

Messy prompts become structured tests

The benchmark contains 369 user requirements and 5,088 acceptance criteria collected from production traffic on an enterprise-internal replica of Code Arena. The team anonymized the requirements, removed duplicates with MinHash and MiniLM-based methods, and had three annotators review the resulting dataset.

The acceptance criteria cover functionality, content, and visual presentation. The high rates of incomplete and colloquial requests reflect the prompts coding agents receive outside tightly controlled benchmark settings.

The leaderboard splits three ways

The researchers generated applications with 16 frontier models, using Codex for the GPT family and Claude Code for the other models. On 197 validated output pairs from an internal arena, IWC-Bench matched human preferences 85.3% of the time. Agreement rose to 98% when the standardized difference between two benchmark scores exceeded 0.75.

Visual quality and usability correlated strongly across model-level averages, with a coefficient of 0.85. The relationship fell to 0.36 across individual applications, so a polished page provided limited evidence that its interactions worked reliably.

Rankings survive a judge swap

Replacing the main judge, Claude-Opus-4.8, with Gemini-3.7-Flash produced a Spearman correlation of 0.982, indicating near-identical model rankings. Of the 120 possible model pairs, 115 retained the same relative order.

Instrumentation succeeded on 99.88% of the 5,904 generated applications. Coverage guidance raised median function coverage from 91.9% to 94.3% and reduced the number of tasks below 90% coverage by nearly 30%.

Coverage establishes that code executed; it does not prove that the resulting behavior was correct. IWC-Bench uses coverage to guide exploration and relies on evidence-based judges to assess usability and requirement compliance.

Reproduction hits a hard limit

The authors are withholding the dataset and source code to prevent data leakage and comply with business-confidentiality requirements. Outside researchers therefore cannot independently rerun the benchmark, inspect the full evaluation implementation, or test additional models under identical conditions.

Tencent Hunyuan maintains related open evaluation work through the ArtifactsBench repository, which includes 1,825 tasks and an evaluation harness. IWC-Bench remains internal.

A practical design for agent testing

IWC-Bench’s central design choice keeps browser exploration separate from requirement scoring. The explorer searches the application without seeing the acceptance criteria, while coverage indicates which paths remain untested. The scoring agents receive the criteria only after the runtime evidence has been collected, reducing the chance that a missed requirement merely reflects an explorer following the rubric poorly.

The current benchmark covers single-turn, text-only, front-end tasks. It does not evaluate back-end integrations, multi-turn development, or multimodal inputs, which limits its use for applications that depend on databases, authentication services, uploaded media, or iterative clarification.

Developers building evaluation systems for coding agents can apply the same structure without reproducing the full benchmark:

• Run generated applications and test reachable behavior instead of relying on source inspection or screenshot similarity.
• Keep exploratory agents blind to acceptance criteria while they gather runtime evidence.
• Use coverage to guide and stop exploration, rather than treating it as proof of correctness.
• Score aesthetics, usability, and requirement alignment separately.
• Retain screenshots, DOM states, and interaction traces so every verdict can cite observable evidence.

For teams selecting a web-generation model, IWC-Bench also shows why a single aggregate score can conceal operational tradeoffs. Models that produce stronger visuals may deliver weaker interactions, while models with less polished output may satisfy more requirements. Runtime testing exposes those differences before generated applications reach users.