Failure Prediction by Confidence Estimation of Uncertainty-Aware Dirichlet Networks

Reliably assessing model confidence in deep learning and predicting errors likely to be made are key elements in providing safety for model deployment, in particular for applications with dire consequences. In this paper, it is first shown that uncertainty-aware deep Dirichlet neural networks provid...

Full description

Saved in:
Bibliographic Details
Published inICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 3525 - 3529
Main Author Tsiligkaridis, Theodoros
Format Conference Proceeding
LanguageEnglish
Published IEEE 06.06.2021
Subjects
confidence
dirichlet
failure prediction
Network architecture
neural network
Neural networks
Predictive models
Reliability
Safety
Signal processing
Task analysis
uncertainty
Online AccessGet full text

Cover

More Information
Summary:Reliably assessing model confidence in deep learning and predicting errors likely to be made are key elements in providing safety for model deployment, in particular for applications with dire consequences. In this paper, it is first shown that uncertainty-aware deep Dirichlet neural networks provide an improved separation between the confidence of correct and incorrect predictions in the true class probability (TCP) metric. Second, as the true class is unknown at test time, a new criterion is proposed for learning the true class probability by matching prediction confidence scores while taking imbalance and TCP constraints into account for correct predictions and failures. Experimental results show our method improves upon the maximum class probability (MCP) baseline and predicted TCP for standard networks on several image classification tasks with various network architectures.
ISSN:2379-190X
DOI:10.1109/ICASSP39728.2021.9414153