Skip to main content
Back

Reveal-IG Fixes AI Explainability by Ditching Raw Inputs for Probability Distributions

Reveal-IG replaces raw input traversal with distributional probes, fixing sign errors and path artifacts in feature attribution for images and tabular data

  • Reveal-IG replaces raw input traversal in Integrated Gradients with a path through probability distributions, eliminating off-manifold artifacts.
  • The method retains completeness -- attributions sum to the total change in expected model output -- while fixing sign errors common in standard IG.
  • Tested on ImageNet (ResNet-50) and California Housing tabular regression, Reveal-IG leads on sign-sensitive attribution metrics.
  • Works for both images (multiscale blurring probes) and tabular data (per-feature uncertainty modeling) within one unified framework.
  • Main tradeoff: requires Monte Carlo sampling at each step and model gradient access, making it more expensive than vanilla IG.
  • Code is open-source at github.com/murphyka/Reveal-IG.

Feature attribution -- the art of explaining why a model made a prediction -- has a dirty secret. The most popular path-based method, Integrated Gradients (IG), walks a straight line from a blank baseline to the real input and accumulates gradients along the way. It is mathematically elegant and satisfies a key property called completeness (all attribution scores sum to the total change in model output). But that straight-line walk passes through nonsensical intermediate inputs -- half-formed images, partially-filled tabular rows -- that the model was never trained on, and it treats every step on that path with equal weight. The result: noisy, sometimes wrong-signed attributions that can mislead rather than explain.

A new paper from Kieran A. Murphy and Shameen Shrestha introduces Reveal-IG, a framework that lifts path attribution out of raw input space entirely and into a space of probability distributions, sidestepping the core pathology while keeping completeness intact.

The Problem With Walking a Straight Line

Integrated Gradients computes feature importance by integrating gradients from a baseline to the actual input. It works by accumulating gradients along a straight path from a user-defined baseline (e.g., a black image or an all-zero vector) to the actual input, and this path integral ensures that attributions satisfy completeness -- they sum up to the difference between the prediction and the baseline prediction.

The trouble is what happens in between. Linear paths between baseline and input may leave the data manifold, resulting in attributions on input directions never encountered during training. Even worse, saliency maps from standard IG tend to be noisy, with significant attribution mass on irrelevant or background features, particularly in vision models. The early part of the path -- near the blank baseline -- contributes equally to the final attribution as the part near the actual input, even though the model is essentially looking at garbage at that stage.