Skip to main content
Back

IRIT Cracks Open Spiking Neural Networks With Formally Guaranteed Explanations

A new paper maps Binary Spiking Neural Networks to causal logic models, enabling SAT/SMT solvers to produce provably sound explanations that outperform SHAP.

Pro

Paper, Full Access
  • First formal XAI framework for SNNs: Binary Spiking Neural Networks are mapped to causal logic models, enabling provably correct explanations via SAT/SMT solvers.
  • Beats SHAP on correctness: Unlike SHAP, the method guarantees no irrelevant features appear in any explanation, a known failure mode of Shapley-based methods.
  • Binary spikes = natural logic: The all-or-nothing firing of spiking neurons maps cleanly to Boolean clauses, making SAT/SMT encoding tractable without approximation.
  • Validated on MNIST: Pixel-level abductive explanations were computed and compared against SHAP, demonstrating formal soundness the heuristic approach cannot provide.
  • Scalability is unproven: The approach works on small networks; whether SAT/SMT solving holds up on large, real-world SNN deployments remains an open question.
  • Timely for neuromorphic AI: As SNNs reach 90-95% energy savings over GPUs and enter safety-critical domains, formal explainability is becoming a deployment requirement.

Spiking Neural Networks (SNNs) are gaining serious traction as the AI world hunts for more energy-efficient alternatives to standard deep learning. Unlike conventional networks, SNNs activate only when a change occurs, mimicking how biological neurons fire, making them far more efficient by performing only meaningful work. SNN-based models can achieve up to 65% lower energy consumption compared to standard deep neural networks on edge devices while maintaining competitive accuracy. But there's a catch that has quietly blocked their adoption in high-stakes settings: nobody could reliably explain what they were doing.

A new paper from researchers at IRIT (Toulouse) tackles this head-on. It provides a causal analysis of Binary Spiking Neural Networks (BSNNs) to explain their behavior, formally defining a BSNN and representing its spiking activity as a binary causal model. The result is the first framework that gives you provably correct, formally verified explanations for SNN decisions, not just statistical guesses.

Why SNNs Have Been a Black Box

Standard explainability tools like SHAP were built for conventional neural networks. Applying them to SNNs is awkward because SNNs communicate through discrete spikes over time, not continuous activations. SNNs are inherently opaque models that need to be complemented with explanations and justifications of their predictions, particularly in critical applications involving a non-expert audience. XAI for SNNs is a sparsely studied research area, and if left unaddressed, this gap could lead to situations where SNNs are methodologically mature for real-world deployment but remain unused because they lack transparency.

The dominant workaround has been to translate SNNs into regular MLPs and then apply SHAP to those. One approach involves translating trained SNNs into an equivalent Multi-Layer Perceptron model, enabling the use of standard post-hoc explanation techniques such as SHAP. But this approximation loses fidelity, and SHAP itself has a deeper problem: existing SHAP scores will necessarily yield misleading information about the relative importance of features for predictions. Concretely, the feature with the largest absolute Shapley value can actually be irrelevant for the prediction.