SaluteDev's GigaCheck Spots AI-Written Passages With 15% Better Accuracy
A new framework borrows the DETR object detection architecture from computer vision to pinpoint AI-written passages inside mixed human-machine documents.
Takeaways
- GigaCheck reframes AI-text span detection as 1D object detection using a DETR-style transformer.
- Backbone is Mistral-7B tuned with LoRA; two heads handle classification and character-level localization.
- Beats RoBERTa baselines by 15% accuracy on RoFT and cuts MSE 3x on RoFT-ChatGPT.
- Hits AvgRec 0.96 / AUROC 0.99 on the MAGE benchmark spanning 27 generators.
- Scaling to 72B backbone hurt accuracy; 7B stays the recommended default.
- Code is open source at github.com/ai-forever/gigacheck, English-only for now.
Spotting AI-generated text is getting harder as generators improve, and it's especially tricky when a document mixes human writing with LLM output. A team at SaluteDev has published GigaCheck, a detector that borrows an idea from computer vision: treat every AI-written passage as an object to be localized, and use a DETR-style transformer to draw a box around it, at the character level instead of pixels.
The framework stacks two heads on a LoRA-fine-tuned Mistral-7B. One is a plain classifier that answers is this document human or machine? The other, the novel piece, is a DN-DAB-DETR module that predicts continuous character-level intervals of LLM text end-to-end, without any of the token-grouping heuristics prior span detectors rely on. The code is open source on GitHub.
Where earlier detectors fall apart
Most binary AI-text classifiers assume a document has a single author. Real usage looks nothing like that: students paste a draft into ChatGPT and rewrite two paragraphs, journalists edit AI drafts, spammers splice generated filler between human sentences. Document-level detectors lack the resolution to catch these mixed cases reliably.
Recent work has leaned on sequence labeling: classify each token or sentence as human or machine, then stitch the labels together with rules to recover a boundary. That approach forces manual post-processing to group tokens into spans. Others, like Sci-SpanDet, work well only on documents with predictable structure (Introduction, Methods, Results, Discussion). GigaCheck aims to be domain-agnostic.