Improving Trust in Safety-Critical AI Systems: Explainable AI and Anomaly Detection Frameworks for Human Safety in Smart Industries
Date:
🧠 Overview
This presentation summarizes my Industrial PhD research at the University of Torino, conducted in collaboration with Rulex Innovation Labs, on developing trustworthy Artificial Intelligence systems for human safety in smart industrial environments.
The thesis investigates how Explainable Artificial Intelligence (XAI) and Visual Anomaly Detection (VAD) can be combined to improve the reliability, transparency, and practical deployment of AI systems operating in safety-critical applications.
Rather than treating explainability and anomaly detection as separate research areas, the work integrates them into a unified framework capable of detecting unexpected situations while simultaneously explaining the reasons behind AI decisions.
🎯 Research Motivation
Artificial Intelligence is increasingly deployed in manufacturing, healthcare, autonomous robotics, and other safety-critical domains.
Despite impressive predictive performance, many deep learning systems still behave as black boxes, making it difficult for engineers and operators to understand:
- why a decision was made,
- whether it should be trusted,
- which image regions influenced the prediction,
- and how to identify model failures.
This thesis addresses these challenges by improving both the interpretability and the trustworthiness of computer vision systems.
🔬 Main Research Contributions
The thesis consists of several complementary research directions.
1. Explainable AI
- Improved LIME Image through a novel stratified sampling strategy
- Proposed ShapBPT, a hierarchical image explanation framework based on Binary Partition Trees
- Evaluated explanation quality using quantitative metrics and human studies
2. Visual Anomaly Detection
Developed multiple anomaly detection frameworks based on deep generative models, including:
- Autoencoders
- Variational Autoencoders (VAE)
- VAE-GAN architectures
The proposed systems enable detection of unexpected situations while requiring only normal training data.
3. Industrial Safety Monitoring
Designed and deployed AI systems for the European project DistriMuSe, focusing on:
- Safe Human–Robot Collaboration
- Industrial anomaly detection
- Real-time safety monitoring
- Explainable decision support
The developed framework combines anomaly detection with visual explanations to assist human operators.
4. Trustworthy AI Evaluation
The thesis also investigates:
- explanation faithfulness,
- localization quality,
- human interpretability,
- robustness of explanation methods,
- reliability of anomaly detection systems.
Special attention is given to identifying situations where AI models appear accurate while relying on misleading visual evidence.
🏭 Applications
The developed methods were evaluated on multiple real-world applications, including:
- Industrial Robotics
- Human–Robot Collaboration
- Manufacturing Inspection
- Medical Image Analysis
- Object Recognition
- Image Classification
- Visual Anomaly Detection
📊 Research Outcomes
The PhD resulted in several international scientific contributions, including:
- AAAI Conference
- ICPE / QualITA Workshop
- XAI World Conference
- Springer Journals
- MDPI Journals
It also produced:
- Open-source software
- Python packages
- Public datasets
- Reproducible research resources
🧪 Demonstrations
The defense included demonstrations of:
- Stratified LIME
- ShapBPT
- Explainable anomaly detection
- Industrial safety monitoring
- Human preference studies
- Real-time robotic safety monitoring
🎯 Impact
This research contributes toward the development of Trustworthy Artificial Intelligence by combining:
- Explainable AI
- Visual Anomaly Detection
- Deep Generative Models
- Human-Centered AI
- Industrial Safety
The proposed methodologies improve transparency, reliability, and practical deployment of AI systems operating in environments where human safety is essential.
Trustworthy AI requires not only accurate predictions, but also understandable and verifiable explanations.
📚 Related Resources
Thesis Page PDF GitHub Google Scholar
🏷️ Keywords
Trustworthy AI · Explainable AI · Anomaly Detection · Visual Anomaly Detection · ShapBPT · LIME · Human-Robot Collaboration · Industrial AI · Deep Generative Models · Computer Vision · Industrial PhD