---
title: "Researchers Expose How Equally Good Models Tell Wildly Different AI Stories"
subtitle: "A new metamorphic testing framework exposes when SHAP and LIME explanations are untrustworthy — without needing ground-truth labels"
url: "https://alphasignal.ai/news/researchers-expose-how-equally-good-models-tell-wildly-different-ai-stories"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-04T00:00:00.000Z"
updated: "2026-06-12T01:36:24.974Z"
type: "Paper"
topics:
  - "Security"
  - "Benchmarks"
subtopics:
  - "Red Teaming"
upvotes: 1303
source_url: "https://arxiv.org/abs/2606.06056"
comments: 0
truncated: true
---

# Researchers Expose How Equally Good Models Tell Wildly Different AI Stories

> A new metamorphic testing framework exposes when SHAP and LIME explanations are untrustworthy — without needing ground-truth labels

You trained five models. They all hit the same validation score. You pick one, run SHAP on it, and present the feature importances to your stakeholders. But here's the uncomfortable truth: had you picked a different model from that equally-good pool, SHAP might have told a completely different story. This is the Rashomon effect in explainable AI , and a new paper from Helge Spieker, Jørn Eirik Betten, and Arnaud Gotlieb proposes a principled way to catch it before it erodes trust in your system.

## The problem no one talks about at model selection time

The Rashomon effect in machine learning describes the situation in which many models fit the training data equally well but tell different stories about which features matter. This isn't a niche edge case , it's the default state of modern ML pipelines, where hyperparameter sweeps and ensembles routinely produce dozens of near-equivalent candidates. When such models are paired with post-hoc explanation methods, stakeholders , end users, regulators, domain experts , may receive conflicting accounts of the same phenomenon depending on which model is selected, undermining trust and actionability.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Metamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning](https://arxiv.org/abs/2606.06056v1)
- [Metamorphic Testing with the Rashomon Set (HTML full paper)](https://arxiv.org/html/2606.06056v1)
- [Test your Machine Learning Algorithm with Metamorphic Testing — Medium](https://medium.com/trustableai/testing-ai-with-metamorphic-testing-61d690001f5c)
- [Assessing Model-Agnostic XAI Methods against EU AI Act Explainability Requirements](https://arxiv.org/abs/2604.09628)
- [How does Explainable AI contribute to regulatory compliance in the EU and US?](https://milvus.io/ai-quick-reference/how-does-explainable-ai-contribute-to-regulatory-compliance-in-the-eu-and-us)