Unsupervised Object Localization with Representer Point Selection

TL;DR

This paper introduces an unsupervised object localization technique that explains model predictions through representer point selection, leveraging pre-trained models without extra fine-tuning, and surpasses existing methods in accuracy.

Contribution

The paper presents a novel unsupervised localization approach based on representer points, providing interpretability and improved performance over prior methods.

Findings

Outperforms state-of-the-art unsupervised localization methods.

Provides interpretability by highlighting important training points.

Achieves superior results even compared to weakly supervised and few-shot methods.

Abstract

We propose a novel unsupervised object localization method that allows us to explain the predictions of the model by utilizing self-supervised pre-trained models without additional finetuning. Existing unsupervised and self-supervised object localization methods often utilize class-agnostic activation maps or self-similarity maps of a pre-trained model. Although these maps can offer valuable information for localization, their limited ability to explain how the model makes predictions remains challenging. In this paper, we propose a simple yet effective unsupervised object localization method based on representer point selection, where the predictions of the model can be represented as a linear combination of representer values of training points. By selecting representer points, which are the most important examples for the model predictions, our model can provide insights into how the model predicts the foreground object by providing relevant examples as well as their importance. Our method outperforms the state-of-the-art unsupervised and self-supervised object localization methods on various datasets with significant margins and even outperforms recent weakly supervised and few-shot methods.