CLIP-CE Loss Beats Dehazing Pipelines for Foggy Object Detection
A new training loss uses CLIP text prompts instead of dehazing modules to rescue object detectors when fog wipes out visual cues.
ProTakeaways
- New CLIP-guided cross-entropy loss boosts object detection in fog without a dehazing preprocessor. Paper
- Approximation of Mutual Exclusion (AME) reweights training samples by how weakened their semantics are.
- Fine-tuned AME (FAME) rebalances the loss adaptively using the detector's own predicted confidence.
- HazyCOCO releases 61,258 synthetic hazy images built from outdoor COCO2017 scenes with depth-based haze.
- Reports state-of-the-art results on hazy detection and generalizes to low-light and underwater conditions.
- Training-time change only, no CLIP inference overhead; code and dataset promised but not yet released.
Detecting cars, pedestrians, and street signs through fog is one of the ugliest edge cases in computer vision. The usual workaround is to bolt a dehazing network in front of the detector and hope the cleaned-up image contains enough signal. A new paper argues that pipeline has the wrong shape, and that a well-designed loss function driven by CLIP text prompts can do the job better without touching pixels.
The authors introduce CLIP-CE, a CLIP-guided cross-entropy loss that reweights the classification objective based on how badly haze has eroded an object's semantics. They claim this is the first work to leverage a vision-language model to boost object detection in hazy environments. Alongside the loss, they release HazyCOCO, a synthetic haze dataset built to be more realistic than what the field has been using.
Why dehazing-then-detecting keeps breaking
Haze attenuates contrast, adds airlight, and quietly erases the mid-level features (edges, textures, part configurations) that detectors were trained to rely on. Objects sitting far from the camera get hit hardest because atmospheric scattering is depth-dependent.
The default fix is a two-stage pipeline: run a dehazing network first, then feed the restored image to a detector. Adverse weather conditions degrade deep learning-based detectors, and image restoration is a common preprocessing step before detection. Building a positive correlation between these two tasks is technically difficult. Dehazing modules are trained on pixel losses like PSNR/SSIM that correlate only loosely with downstream detection accuracy, and when they misfire they can actively hurt the detector.