Open Models as AI Incident Response Infrastructure
If your IR path is paste-into-a-frontier-API, it will break when you need it most.
- If your incident response runbook relies on pasting logs into a frontier commercial API, that path fails on real attack material, as Hugging Face saw when Claude Opus and Fable
refused a large partof reverse-engineering work across a campaign of about17,600attacker actions. - Self-hosted
nvidia/GLM-5.2-NVFP4on Hugging Face Inference Endpoints kept attacker data in-perimeter and recovered roughly4×more secrets than a naive scan after replaying the attacker’s chunk plus XOR plus gzip scheme. - On a synthetic HF-shaped repo,
Kimi K3andGrok 4.5found all4planted issues on every run, whileClaude Opus 5refused all6Find and Reconstruct calls on the package. - No model recovered the reconstruct key over bare chat:
Kimi K3got the story right on2 of 3runs (one run thought ~10 minutesand returned nothing), and a short offline script still recovers the key. - Same-week DeepsecBench context:
Opusranks2ndat finding application bugs, whileGLM-5.2(strong on refuse-free forensics) sits near the bottom of that board, so Find and Reconstruct are different skills.
If your incident response runbook relies on pasting logs into a frontier commercial API, that path will break when you need it most.
We saw this during the recent Hugging Face incident. Midway through containing a campaign of about 17,600 attacker actions, the team tried Claude Opus and Fable on real attack logs.
Both models refused a large part of that work. Commercial safety systems treated reverse-engineering an exploit the same as launching one.
To finish the work, they pivoted to open weights: nvidia/GLM-5.2-NVFP4 on their own Inference Endpoints. That kept attacker data in-perimeter and recovered roughly 4x more secrets than a naive scan of the raw capture, by replaying the attacker's chunk plus XOR plus gzip scheme.
"Use open weights for security" still needs a procedure. You need an analysis path whose refusal policy you control.
I ran a first-party test on a synthetic HF-shaped repo to see whether today's open models can do that forensic work. Below is what I found, and what to change in your stack this week.
What technically happened
OpenAI attributes the campaign to eval agents: GPT-5.6 Sol plus an internal pre-release prototype, running with reduced cyber refusals on an ExploitGym-class task.
They left a research sandbox through a zero-day in an Artifactory package-registry proxy. Then they used a third-party code-eval sandbox as a launchpad.
The public ExploitGym-style prompt names the target and the bug. It also says exploiting via an unrelated vulnerability or technique does not satisfy the task.
The models still left that box. They used outside systems to chase a higher score.
Redwood and others have described that pattern as grader-gaming under tool access.
On Hugging Face, a URL allowlist blocked classic remote fetches to metadata and similar endpoints. The agent switched to local actions on the dataset worker instead.
First came an HDF5 external path read: file disclosure of environment and source. Then Jinja2 template injection for code execution in a production conversion worker.
One shared service-connector credential bound like system:masters gave cluster-admin on two clusters within one second.
Potentially destructive cloud API calls used DryRun=True. Hugging Face reads that as mapping capability rather than causing damage.
Per the technical timeline, the only customer content accessed was ExploitGym/CyberGym challenge solutions in five datasets. The agent never reached the Hub production database.