Open-Source DiscoveryRSI-27B Beats GPT-6 at Predicting Future Scientific Research
A 27B open model learns to generate scientific hypotheses recursively, matching real published research at nearly triple the rate of GPT-6 and Claude Opus 5.
- DiscoveryRSI-27B hits a 6.3% match rate with research published after its training cutoff, versus 2.5% for GPT-6 and 2.3% for Claude Opus 5.
- Three recursive learning mechanisms (credit optimization, harness replay, rediscovery) lift hypothesis quality from 4.86 to 5.39.
- Hypotheses are versioned objects with claims, investigation plans, decision criteria, and alternative explanations.
- Compiled 50% more executable Harbor tasks than its base model in matched runs.
- DFM-Bench scores research at five levels: hypothesis, transition, trajectory, campaign, and discovery.
- Open Apache-2.0 harness and 27B checkpoint available, with a live arena streaming running campaigns.
DiscoveryRSI is an open-source research system that generates, tests, and revises scientific hypotheses. Its 27B-parameter model achieved a 6.3% match rate with research published after its training cutoff, compared with 2.3% to 5.4% for the listed competitors. The accompanying paper presents this retrospective test as evidence that the system can identify research directions later pursued by scientists.
The project’s main contribution is a feedback loop built around inspectable research records. Each hypothesis carries its evidence, investigation plan, decision criteria, alternatives, and revision history. That structure lets the system learn from the steps that improved a research campaign and gives evaluators more to inspect than polished prose.
Hypotheses become research records
General-purpose language models often produce scientific claims that are difficult to falsify, weakly connected to available data, or dependent on controls and experiments that researchers cannot run. Fluent writing can conceal those defects, making output-level grading unreliable.
DiscoveryRSI records every proposal as a versioned object containing:
- An evidence-grounded claim.
- An investigation plan tied to available data and tools.
- Explicit criteria for accepting, rejecting, or revising the claim.
- Alternative explanations for the expected result.
- A lineage showing how the proposal changed over time.
The version history allows evaluators to assign credit to specific edits, such as adding a comparator, clarifying a measurement, tightening a falsification rule, or repairing an infeasible study design.
Execution traces drive the next round
A campaign begins with multi-agent evolutionary search that proposes and branches hypotheses. The framework selects promising investigations, compiles them into executable tasks through the Harbor harness, and records the resulting evidence, failures, and critiques.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
