A3Rank: Augmentation Alignment Analysis for Prioritizing Overconfident Failing Samples for Deep Learning Models

TL;DR

This paper introduces A3Rank, a novel test case prioritization method that uses augmentation alignment analysis to identify high-confidence failing samples in deep learning models, outperforming existing techniques.

Contribution

A3Rank is a new technique that effectively ranks failing samples by analyzing prediction misalignments with augmented test cases, enhancing detection beyond confidence-based rejectors.

Findings

A3Rank outperforms peer techniques by 163.63% in detection ratio.

The framework improves defense success rate against high-confidence failures.

Augmentation alignment analysis effectively identifies elusive failing samples.

Abstract

Sharpening deep learning models by training them with examples close to the decision boundary is a well-known best practice. Nonetheless, these models are still error-prone in producing predictions. In practice, the inference of the deep learning models in many application systems is guarded by a rejector, such as a confidence-based rejector, to filter out samples with insufficient prediction confidence. Such confidence-based rejectors cannot effectively guard against failing samples with high confidence. Existing test case prioritization techniques effectively distinguish confusing samples from confident samples to identify failing samples among the confusing ones, yet prioritizing the failing ones high among many confident ones is challenging. In this paper, we propose $A^3$Rank, a novel test case prioritization technique with augmentation alignment analysis, to address this problem. $A^3$Rank generates augmented versions of each test case and assesses the extent of the prediction result for the test case misaligned with these of the augmented versions and vice versa. Our experiment shows that $A^3$Rank can effectively rank failing samples escaping from the checking of confidence-based rejectors, which significantly outperforms the peer techniques by 163.63\% in the detection ratio of top-ranked samples. We also provide a framework to construct a detector devoted to augmenting these rejectors to defend these failing samples, and our detector can achieve a significantly higher defense success rate.