Robustness and Overfitting Behavior of Implicit Background Models

TL;DR

This paper investigates how implicit background models in image classification overfit and proposes an overfit detection method using segmentation masks, along with strategies to reduce overfitting through data augmentation.

Contribution

It introduces a novel overfit detection criterion based on segmentation masks and evaluates overfitting reduction techniques in the context of implicit background models.

Findings

Overfitting can be effectively detected without test labels.

Data augmentation improves model calibration and reduces overfitting.

Segmentation masks provide valuable insights into model overfitting behavior.

Abstract

In this paper, we examine the overfitting behavior of image classification models modified with Implicit Background Estimation (SCrIBE), which transforms them into weakly supervised segmentation models that provide spatial domain visualizations without affecting performance. Using the segmentation masks, we derive an overfit detection criterion that does not require testing labels. In addition, we assess the change in model performance, calibration, and segmentation masks after applying data augmentations as overfitting reduction measures and testing on various types of distorted images.