Sensitivity analysis of image classification models using generalized polynomial chaos

TL;DR

This paper applies generalized polynomial chaos to perform sensitivity analysis on image classification models, quantifying how input uncertainties and domain shifts affect model outputs, validated through a welding defect classification case study.

Contribution

It introduces a novel approach using GPC and Sobol indices to analyze the impact of input uncertainties on image classification models.

Findings

GPC effectively quantifies input influence on model outputs.

Sensitivity analysis reveals key input parameters affecting classification accuracy.

Method validated on real-world welding defect classification data.

Abstract

Integrating advanced communication protocols in production has accelerated the adoption of data-driven predictive quality methods, notably machine learning (ML) models. However, ML models in image classification often face significant uncertainties arising from model, data, and domain shifts. These uncertainties lead to overconfidence in the classification model's output. To better understand these models, sensitivity analysis can help to analyze the relative influence of input parameters on the output. This work investigates the sensitivity of image classification models used for predictive quality. We propose modeling the distributional domain shifts of inputs with random variables and quantifying their impact on the model's outputs using Sobol indices computed via generalized polynomial chaos (GPC). This approach is validated through a case study involving a welding defect classification problem, utilizing a fine-tuned ResNet18 model and an emblem classification model used in BMW Group production facilities.