Uncertainty Measurement of Deep Learning System based on the Convex Hull of Training Sets

TL;DR

This paper introduces a novel uncertainty measurement method for deep learning models based on the convex hull of training data, enabling better detection of out-of-distribution samples and adversarial attacks.

Contribution

It proposes To-hull Uncertainty and Closure Ratio, new metrics that quantify how far unseen samples are from the training data convex hull, improving uncertainty estimation.

Findings

Effective in detecting samples with unusual patterns

Outperforms existing test selection metrics in experiments

Identifies adversarial attacks more accurately

Abstract

Deep Learning (DL) has made remarkable achievements in computer vision and adopted in safety critical domains such as medical imaging or autonomous drive. Thus, it is necessary to understand the uncertainty of the model to effectively reduce accidents and losses due to misjudgment of the Deep Neural Networks (DNN). This can start by efficiently selecting data that could potentially malfunction to the model. Traditionally, data collection and labeling have been done manually, but recently test data selection methods have emerged that focus on capturing samples that are not relevant to what the model had been learned. They're selected based on the activation pattern of neurons in DNN, entropy minimization based on softmax output of the DL. However, these methods cannot quantitatively analyze the extent to which unseen samples are extrapolated from the training data. Therefore, we propose To-hull Uncertainty and Closure Ratio, which measures an uncertainty of trained model based on the convex hull of training data. It can observe the positional relation between the convex hull of the learned data and an unseen sample and infer how extrapolate the sample is from the convex hull. To evaluate the proposed method, we conduct empirical studies on popular datasets and DNN models, compared to state-of-the art test selection metrics. As a result of the experiment, the proposed To-hull Uncertainty is effective in finding samples with unusual patterns (e.g. adversarial attack) compared to the existing test selection metric.