Researchers Argue XAI Should Be Embedded in Engineering Simulation Workflows
A new survey maps explainable AI techniques onto every stage of surrogate modeling, turning opaque simulation emulators into transparent tools for engineering decisions.
Takeaways
- Survey unifies surrogate modeling and XAI, two fields that have evolved separately despite obvious complementarity.
- Maps global sensitivity, interaction visualization, and local attribution methods onto engineering simulation workflows.
- Diagnoses why standard XAI methods break on correlated inputs, mixed variables, and dynamical systems.
- Advocates multi-method deployment because Rashomon effect and no-free-lunch make single-explainer trust risky.
- Identifies uncertainty quantification of explanation metrics as an urgent research direction using bootstrapping or conformal prediction.
- Argues explainability must be embedded from design-of-experiments through decision support, not bolted on post-hoc.
Surrogate models have quietly become the backbone of modern engineering simulation. When a computational fluid dynamics run takes hours or an agent-based epidemic model burns through a compute cluster, teams train a fast machine learning emulator to stand in for the real thing. The catch is that these emulators inherit the black-box nature of what they replace, and often amplify it. A new survey from a team spanning ITB, IRIT Toulouse, ISAE-SUPAERO, and Kyushu University argues that this is exactly where explainable AI should live, and lays out a concrete workflow for making it happen.
Two fields that should have met years ago
Surrogate modeling and XAI have evolved as disjoint but highly complementary areas of research. Surrogates deliver speed. XAI delivers transparency. Standard interpretability methods, however, were built for tabular ML tasks and tend to break on the constraints that define engineering work: highly correlated inputs, mixed-variable design spaces, dynamical systems, or the need for rigorous uncertainty quantification.
The authors position the two paradigms as mutually reinforcing. XAI gives surrogates the diagnostic layer engineers need to trust them, and well-constructed emulators provide the smooth, noise-controlled mathematical space where interpretability metrics can actually be computed stably.
A workflow, not another taxonomy
Rather than cataloging XAI algorithms in isolation, the survey maps them directly onto the stages of a simulation-driven workflow. The pipeline goes from simulations to surrogates, then across a spectrum of explainability methods, and finally into engineering insights fed back into design. The framing rejects the usual global-vs-local dichotomy in favor of a multi-scale diagnostic approach.