Tencent's WorkForge Trains AI Agents on Real Files, Beating Models 15x Larger

Training agents for professional knowledge work requires workspaces that demand file discovery, cross-checking, and sustained tool use. Building each environment by hand consumes substantial time, and fully synthetic environments often produce small, obvious file sets. Researchers at Tencent’s LLM Department and Fudan University propose WorkForge, a framework that gathers repositories and datasets from the web, organizes them into task workspaces, and derives grading criteria from evidence in those files.

Across 40 professional domains, the framework generated 16,665 verifiable environments containing varied file types and dependencies between documents. Supervised fine-tuning on trajectories collected in these workspaces raised scores for a 35-billion-parameter Qwen model on two professional-agent benchmarks. Tool logs also showed broader web searches, more targeted evidence retrieval, and longer diagnostic responses after failed calls.

Real files expose synthetic shortcuts

Earlier environment-synthesis systems commonly generate mock resources from scratch. In a controlled comparison using the same expert-authored blueprints, WorkForge environments built from real resources contained 5.4 times as many files and 2.4 times as many file types as their synthetic counterparts. Their average size increased from 21.5 KB to 279.0 KB.

The real-resource workspaces also distributed relevant evidence across directories and file formats. Agents had to inspect the workspace, identify relationships between files, and reconcile values before producing an answer. An obviously named file containing every required fact could no longer support the task.

Ungrounded verification creates a separate problem. A language-model judge can invent requirements when its rubric lacks support in the workspace, potentially rewarding fabricated content. WorkForge generates tasks and checks from the same workspace profile, so each requested value or judgment should trace to an observable file, dependency, or factual anchor.

Four stages preserve the evidence trail

WorkForge converts an expert workflow into scored agent trajectories through four stages, with structured outputs passing between them:

1. Workflow specification. Domain experts create reusable Scene Blueprints describing the workflow, required resources, decision points, and expected deliverables. A blueprint for grading a unit test, for example, can be instantiated with different course materials.
2. Environment synthesis. An acquisition agent searches the web, downloads repositories and datasets, and arranges them into a workspace. A profiling pass catalogs each file’s role, records dependencies, and extracts factual anchors that later support tasks and graders.
3. Task generation. The system uses the blueprint and workspace profile to produce the instruction shown to the agent, a hidden solution plan, deterministic checks, and semantic rubrics. The hidden plan guides task construction and evaluation without revealing the solution to the solver.
4. Trajectory generation. A solver built with the Claude Code or OpenClaw harness and driven by GLM 5.3 attempts the task. Its sequence of actions, tool calls, intermediate responses, and submitted artifacts forms a trajectory that the verifiers score for training.

Programmatic graders handle exact, machine-checkable facts, such as whether a CSV identifies 198907 as the start date for an Emerging five-factor series. Semantic graders assess qualities that resist exact matching, such as whether a briefing deck reconciles regional differences in financial premia. The paper’s ablation study found that the combined verifier outperformed configurations using either grader type alone.

A corpus built for long tool runs

The generated corpus combines large workspaces with lengthy interactions and detailed evaluation criteria:

Office Productivity, Data & Database, Business & Enterprise, and Finance & Payments account for roughly 32% of tasks. The remaining coverage extends to fields including medicine, cybersecurity, energy, and logistics.

Fine-tuning lifts two agent benchmarks

The authors used all 16,665 scored trajectories for supervised fine-tuning, which trains a model to reproduce successful action sequences and outputs. On Qwen3.5-35B-A3B-Base, WorkForge training raised GDPval by 28.1 points and APEX by 16.3 points:

The 73.6 GDPval score exceeded every open model below 100 billion parameters in the authors’ comparison, including Nemotron-Terminal-32B, ClawGym-30A3, EnvScaler-8B, and OpenThinkerAgent-32B.

WorkForge data also improved every reported benchmark when applied to Qwen3.5-27B, a model that had already undergone post-training to shape its task behavior. The resulting model scored 82.4 on GDPval, above Qwen3.5-397B-A17B at 79.7 while using about one-fifteenth as many total parameters. It remained below Claude Opus 4.8 at 89.1 and GPT-5.6 in the authors’ comparison.

Tool logs show broader search and longer recovery

Behavioral measurements on GDPval indicate that the fine-tuned 27B model changed how it approached tasks, including what it searched and how it responded to tool failures:

The longer post-failure responses provide a measurable proxy for diagnosis and replanning, although they do not prove that every additional token improves recovery. Rubric-level results show the largest gains in formatting, up 13 points, and numerical accuracy, up 8.2 points.

Returns continue through 16K trajectories

Training-set scaling remained monotonic across the tested range from 1,000 to 16,000 trajectories on ALE and WildClaw. The model trained on 16,000 trajectories also surpassed DeepSeek V3.2 on ALE, suggesting that the pipeline had not exhausted its gains at the largest reported dataset size.

A separate experiment grouped training trajectories into 0–20, 21–40, and 41–60-turn ranges. Training with the longest group yielded an average of 24.0 productive tool calls on WildClaw, compared with 11.7 for the shortest group, and produced a broader mix of tool types. The result supports including sustained, multi-step interactions when training agents for long tasks.

Reproduction still requires missing pieces

The framework depends on expert-authored workflows and web-accessible resources, which constrains coverage of proprietary, confidential, and sensitive domains. Its verifiers score submitted artifacts without directly grading the quality of intermediate behavior, so an inefficient or error-prone trajectory can pass when its final deliverable satisfies the checks.

At publication, the WorkForge paper and Hugging Face entry did not link to public code or dataset files. Reproduction therefore requires rebuilding the acquisition, profiling, task-generation, verification, and solver components from the published method description.

A production implementation would also need controls for source licensing, duplicate content, benchmark contamination, and untrusted downloaded files. Sandboxing the acquisition and solver processes is especially relevant because the pipeline ingests repositories and documents from external sources.

Teams can apply the core design without the original implementation by profiling workspace files first, recording factual anchors and dependencies, deriving instructions and graders from those records, and retaining solver trajectories with artifact-level scores. Source governance and evaluation of intermediate behavior remain separate engineering requirements.