Repairing Group-Level Errors for DNNs Using Weighted Regularization

TL;DR

This paper introduces Weighted Regularization (WR), a novel approach to repair class confusion and bias errors in DNNs, improving their reliability in decision-making tasks without significantly impacting overall performance.

Contribution

The paper presents a generic WR method that effectively repairs group-level errors in DNNs, addressing confusion and bias errors for both single-label and multi-label classifications.

Findings

WR significantly reduces confusion errors across datasets.

WR methods outperform baseline approaches in error mitigation.

Limited impact on overall model performance.

Abstract

Deep Neural Networks (DNNs) have been widely used in software making decisions impacting people's lives. However, they have been found to exhibit severe erroneous behaviors that may lead to unfortunate outcomes. Previous work shows that such misbehaviors often occur due to class property violations rather than errors on a single image. Although methods for detecting such errors have been proposed, fixing them has not been studied so far. Here, we propose a generic method called Weighted Regularization (WR) consisting of five concrete methods targeting the error-producing classes to fix the DNNs. In particular, it can repair confusion error and bias error of DNN models for both single-label and multi-label image classifications. A confusion error happens when a given DNN model tends to confuse between two classes. Each method in WR assigns more weights at a stage of DNN retraining or inference to mitigate the confusion between target pair. A bias error can be fixed similarly. We evaluate and compare the proposed methods along with baselines on six widely-used datasets and architecture combinations. The results suggest that WR methods have different trade-offs but under each setting at least one WR method can greatly reduce confusion/bias errors at a very limited cost of the overall performance.