When to Gate Recursive Terminal-task Synthesis

A training job for a coding agent that lives in a real shell is four pieces that have to agree: a public instruction, a workspace, an official solution, and hidden tests.

If they don't, the official solution fails in the container, or the hidden tests check a file the agent was never told to create.

Hand-writing that bundle often costs hundreds to thousands of dollars per task, per Recursive Synthesis for Long-Horizon Terminal Tasks.

RST (Recursive Synthetic Terminal Tasks) is the loop a Tencent Hunyuan-led team built to manufacture those jobs without writing each one by hand.

The loop takes one verified terminal task, grows a harder child with more real work in the repo and tests rewritten to match, then uses that child as the next parent.

Promote means auto-generate, because the job can be made hermetic, bounded, reproducible, and outcome-verifiable.

Hybrid means humans still pick domains, define done, and audit tests.

Reject means keep the job out.

That loop can manufacture the practice set, and it cannot replace the human-authored exam.

Harbor is the open eval harness that runs an agent in a container against a public instruction, a reference solution, and hidden tests, which is already how Claude Code, Codex, and OpenHands get scored.

We asked the Tencent Hunyuan authors how those four pieces fail, and they named three breakages, the solution not running in the container, the verifier checking an artifact the solution never creates, or a hidden requirement that cannot be inferred from the public workspace.

"One inconsistency can invalidate the entire task." Zhongzhi Li, correspondence with AlphaSignal.

What follows is the four-word test on three real jobs, the JSON-diff ordering, the validity-boundary line items, and the volume/authority split.

How do you make a terminal task harder?

You make a terminal task harder by growing the official solution and the environment, then rewriting the hidden tests to match the new work, then rewriting the public instruction last.

A useful Harbor item is one of those four-part packages. Those four have to agree, or the whole example is noise.

RST keeps an executable proof of solvability while adding work in the workspace, and it splits acceptance in two in the paper's preliminaries.

Oracle-valid means the official reference solution, solve.sh, passes the private verifier in a fresh sandbox.

solve.sh is the script that ships with the task and shows one way to finish the job. The factory grows that script first, then writes tests that this script can pass.

Contract-valid is the other half, because every semantic check is stated in the public instruction or discoverable from the workspace. A bundle that is only oracle-valid can still train the agent on a private literal.

Then RST rewrites the tests with the solution. The copied order is transformation contract, grow solve.sh and the local environment, derive verifier checks from observable artifacts, rewrite the public instruction last, then at most two constrained repair rounds.

JSON-diff is that ordering on one lineage. The seed is a 10-line reference solution that runs gendiff on fixed JSON pairs.