Can I Trust My Anomaly Detection System? A Case Study Based on Explainable AI
Date:
🧠 Overview
This talk presents a case study on the trustworthiness of anomaly detection systems, focusing on models based on Variational Autoencoders (VAEs).
While reconstruction-based anomaly detection methods often achieve high accuracy, their decision process remains opaque and potentially misleading, especially when anomaly scores are influenced by irrelevant or spurious features.
🔍 Key Idea
Instead of relying only on anomaly scores, this work asks:
Can we trust what anomaly detectors are actually learning?
Using Explainable AI (XAI) techniques, the study analyzes the internal behavior of VAE-based models to understand why certain samples are classified as anomalous.
⚠️ Key Findings
- High anomaly scores do not always correspond to true anomalies
- Models may rely on irrelevant visual patterns
- Reconstruction-based methods can produce misleading explanations
- Trust in anomaly detection requires interpretability, not only performance
🔗 Resources
🧪 Methodology
- Model: Variational Autoencoder (VAE)
- Task: Visual anomaly detection
- Goal: Explain why reconstruction-based anomaly detectors flag samples as anomalous
- Analysis: Reconstruction error versus explanation maps
🎯 Impact
This work highlights a key limitation of reconstruction-based anomaly detection:
Accurate anomaly detection does not necessarily imply trustworthy decision-making.
It motivates the integration of Explainable AI into anomaly detection pipelines, especially for safety-critical and industrial applications.
🏷️ Keywords
Explainable AI · Anomaly Detection · VAE · Trustworthy AI · Feature Attribution · Industrial AI