University of Sydney's RankOOD Beats 34 AI Safety Baselines by Catching Overconfident Predictions
A new out-of-distribution detection method uses class ranking patterns and the Plackett-Luce loss to catch inputs that overconfident classifiers would otherwise miss.
PRO- RankOOD uses class ranking patterns and the Plackett-Luce loss to detect out-of-distribution inputs without outlier exposure.
- Achieves state-of-the-art near-OOD detection on TinyImageNet, reducing FPR95 by 4.3% over the strongest baseline.
- Beats 34 methods in the OpenOOD benchmark across CIFAR-10, CIFAR-100, and ImageNet-200.
- Three-stage pipeline: extract canonical ranks via ILP, retrain with ListMLE plus cross-entropy, score deviations at inference.
- Works on standard ResNet backbones with no architectural changes, unlike prior rank-based methods like CRAFT.
- Costs about 34% more GPU time per epoch than LogitNorm; no public code release yet. Read the paper.
Deep neural networks have a nasty habit of being confidently wrong. Show a classifier trained on animals a picture of a car, and it will happily tell you it is 98% sure that car is a golden retriever. That failure mode, known as out-of-distribution (OOD) detection, is one of the main reasons deep learning models still cannot be trusted in safety-critical settings like autonomous driving or medical imaging.
A team from the University of Sydney and the Qatar Computing Research Institute has proposed RankOOD, a training-based OOD detection method that reframes the problem as a ranking task. Instead of asking whether the top-1 prediction is correct, the method asks whether the entire ordering of class scores matches what an in-distribution sample would produce.
Ranking beats top-1 guessing
A DNN classifier trained for single-class prediction often induces a rank order across classes. An OOD example may be overconfidently assigned to a class label but is unlikely to respect the associated rank order. When a cross-entropy trained model sees a real cat image, the logits for cat rank first, then dog, then other four-legged animals, and so on down to airplane. That ordering encodes semantic similarity. OOD inputs might spike a single logit high, yet they rarely reproduce the full descending pattern.
Prior work like ExCeL and CRAFT tried to exploit this signal, but modeled ranks as probability mass function matrices and required architectural changes. RankOOD instead operates directly on the raw logits of a standard vision backbone.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
