Mira Murati's Thinking Machines Pays Researchers $50K to Make Open Weights Safer

Thinking Machines Lab, the AI startup led by former OpenAI CTO Mira Murati, has opened a funding program aimed at one of the trickier corners of AI research: how to release model weights publicly without also handing attackers a loaded weapon. The lab is offering Tinker grants of up to $50,000 in credits for safety research on open-weight models, and the project ideas published alongside the announcement read like a technical wish list for the field.

The grants plug directly into Tinker, the company's fine-tuning API. Tinker is a Python-based interface that lets researchers fine-tune open-weight models ranging from 1B to 1T+ parameters, covering both dense and mixture-of-experts architectures. Users write a training loop locally, and Thinking Machines' infrastructure handles the distributed GPU work behind the scenes. Credits are the currency researchers burn to run those jobs, so $50,000 buys a serious amount of experimentation.

Why fund this now

The timing follows Inkling, a recently released model that reasons natively over text, images, and audio, trained as a broad foundation model and shipped with a smaller sibling. Both have open weights. Once weights are out in the world, the safety story changes: any downstream user can fine-tune the model, strip guardrails, or push it into domains its creators never intended. The grants program is essentially the lab crowdsourcing answers to the questions its own release strategy raises.

What the lab actually wants built

The proposal document lays out three research pillars. The first, making models safer to open, has three sub-directions worth pulling apart:

• Differential defense over offense. Many dangerous skills are dual-use. The idea is to fine-tune models so they get meaningfully better at defensive tasks like triage, detection, and hardening without proportionally boosting exploitation or evasion. One suggested experiment: measure the offense-defense gap of a single training run, not just raw capability.
• Hazardous data classifiers at scale. Can filters catch safety-relevant content in pretraining data without discarding huge swaths of legitimate scientific writing? And can those filters survive paraphrasing, obfuscation, and shifting model capabilities?
• Tamper-resistant safety training. Safeguards baked in during training often wash out under later fine-tuning. The lab wants methods that persist under both adversarial fine-tuning and ordinary continued training.

The second pillar targets alignment failure modes, particularly what they call safety-relevant generalization from narrow fine-tuning. This nods to emergent misalignment, a phenomenon where training a model on a narrow harmful task somehow produces broadly misaligned behavior across unrelated tasks. Thinking Machines wants to characterize when this happens, whether it requires adversarial data, and whether models can compose separately-learned skills like planning, tool use, and domain knowledge into concerning end-to-end behaviors that were never directly demonstrated in training.

Reward hacking sits alongside it: when a model optimizes a proxy reward well enough to game the evaluation without accomplishing the real objective. The harder version of this problem is oversight gaming, where models learn to interfere with chain-of-thought monitors or other supervisory tools.

Measuring what you can't easily see

The third pillar covers quantifying risk after release. Two questions stand out. Worst-case estimation asks how much a motivated attacker can degrade safeguards given time, data, and compute. Marginal risk asks what a specific model enables beyond what was already possible with existing tools. Adversarial fine-tuning gets treated as the natural stress test, because it separates capabilities that are genuinely absent from ones that are merely suppressed and easy to re-elicit.

The lab is also asking whether safety-relevant properties can be forecast from small runs, so expensive experiments can be planned intelligently rather than run blindly at full scale. That has practical value beyond safety: if capability uplift, safeguard degradation, or reward hacking follow predictable scaling curves, researchers can flag problems before spending a fortune to confirm them.

What this means for the ecosystem

Open-weight safety research has a chronic funding problem. The people best equipped to do it, academic labs and independent researchers, usually cannot afford the compute needed to fine-tune frontier-scale models. Meanwhile, the labs with the compute have obvious incentives against publishing results that make their own releases look risky. A credit-based grant tied to a fine-tuning API is a reasonable workaround: it puts real compute into the hands of outside researchers without requiring them to build infrastructure or negotiate hardware access.

Applicants submit a proposal describing the project and the credits they need. The lab has published application requirements and terms on its site. The list of research directions is explicitly non-exhaustive, so proposals outside the three pillars are on the table as long as they connect to open-weight model safety.