Bridging the Gap between Model Explanations in Partially Annotated Multi-label Classification

TL;DR

This paper investigates how false negative labels in partially annotated multi-label classification affect model explanations and proposes a method to improve explanation quality and classification performance by boosting attribution scores.

Contribution

It introduces a novel approach to enhance explanations in partially labeled multi-label classification by adjusting attribution scores, leading to significant performance improvements.

Findings

Model explanations are similar but scaled differently between full and partial labels.

Boosting attribution scores improves explanation similarity.

Classification performance significantly improves across multiple datasets.

Abstract

Due to the expensive costs of collecting labels in multi-label classification datasets, partially annotated multi-label classification has become an emerging field in computer vision. One baseline approach to this task is to assume unobserved labels as negative labels, but this assumption induces label noise as a form of false negative. To understand the negative impact caused by false negative labels, we study how these labels affect the model's explanation. We observe that the explanation of two models, trained with full and partial labels each, highlights similar regions but with different scaling, where the latter tends to have lower attribution scores. Based on these findings, we propose to boost the attribution scores of the model trained with partial labels to make its explanation resemble that of the model trained with full labels. Even with the conceptually simple approach, the multi-label classification performance improves by a large margin in three different datasets on a single positive label setting and one on a large-scale partial label setting. Code is available at https://github.com/youngwk/BridgeGapExplanationPAMC.