Knowing the Unknown: Interpretable Open-World Object Detection via Concept Decomposition Model

TL;DR

This paper introduces IPOW, an interpretable open-world object detection framework that decomposes features into concepts for better unknown object detection and interpretability.

Contribution

It proposes a Concept Decomposition Model (CDM) for OWOD, enabling explicit feature decomposition and improved detection of unknown objects with interpretability.

Findings

Significantly improves unknown object recall

Reduces known-unknown confusion

Provides concept-level interpretability

Abstract

Open-world object detection (OWOD) requires incrementally detecting known categories while reliably identifying unknown objects. Existing methods primarily focus on improving unknown recall, yet overlook interpretability, often leading to known-unknown confusion and reduced prediction reliability. This paper aims to make the entire OWOD framework interpretable, enabling the detector to truly "knowing the unknown". To this end, we propose a concept-driven InterPretable OWOD framework(IPOW) by introducing a Concept Decomposition Model (CDM) for OWOD, which explicitly decomposes the coupled RoI features in Faster R-CNN into discriminative, shared, and background concepts. Discriminative concepts identify the most discriminative features to enlarge the distances between known categories, while shared and background concepts, due to their strong generalization ability, can be readily transferred to detect unknown categories. Leveraging the interpretable framework, we identify that known-unknown confusion arises when unknown objects fall into the discriminative space of known classes. To address this, we propose Concept-Guided Rectification (CGR) to further resolve such confusion. Extensive experiments show that IPOW significantly improves unknown recall while mitigating confusion, and provides concept-level interpretability for both known and unknown predictions.