Explaining Object Detectors via Collective Contribution of Pixels

TL;DR

This paper introduces a game-theoretic explanation method for object detectors that captures both individual pixel contributions and their collective influence, improving interpretability.

Contribution

It proposes a novel approach using Shapley values and interactions to better explain object detection by considering pixel groups, unlike prior pixel-only methods.

Findings

More accurate identification of important regions than existing methods

Effective explanations for both localization and class determination

Demonstrated superiority through extensive experiments

Abstract

Visual explanations for object detectors are crucial for enhancing their reliability. Object detectors identify and localize instances by assessing multiple visual features collectively. When generating explanations, overlooking these collective influences in detections may lead to missing compositional cues or capturing spurious correlations. However, existing methods typically focus solely on individual pixel contributions, neglecting the collective contribution of multiple pixels. To address this limitation, we propose a game-theoretic method based on Shapley values and interactions to explicitly capture both individual and collective pixel contributions. Our method provides explanations for both bounding box localization and class determination, highlighting regions crucial for detection. Extensive experiments demonstrate that the proposed method identifies important regions more accurately than state-of-the-art methods. The code is available at https://github.com/tttt-0814/VX-CODE