Learning from Pseudo-labeled Segmentation for Multi-Class Object Counting

TL;DR

This paper introduces a novel approach for multi-class object counting by localizing objects with pseudo-labeled segmentation masks derived from minimal annotations, significantly improving performance on new benchmarks.

Contribution

It proposes a pseudo-labeling method for segmentation masks using only box and dot annotations, enabling effective multi-class object localization and counting.

Findings

Outperforms previous CAC methods on new benchmarks

Effective localization of objects using pseudo segmentation masks

Introduces synthetic and real multi-class counting datasets

Abstract

Class-agnostic counting (CAC) has numerous potential applications across various domains. The goal is to count objects of an arbitrary category during testing, based on only a few annotated exemplars. In this paper, we point out that the task of counting objects of interest when there are multiple object classes in the image (namely, multi-class object counting) is particularly challenging for current object counting models. They often greedily count every object regardless of the exemplars. To address this issue, we propose localizing the area containing the objects of interest via an exemplar-based segmentation model before counting them. The key challenge here is the lack of segmentation supervision to train this model. To this end, we propose a method to obtain pseudo segmentation masks using only box exemplars and dot annotations. We show that the segmentation model trained on these pseudo-labeled masks can effectively localize objects of interest for an arbitrary multi-class image based on the exemplars. To evaluate the performance of different methods on multi-class counting, we introduce two new benchmarks, a synthetic multi-class dataset and a new test set of real images in which objects from multiple classes are present. Our proposed method shows a significant advantage over the previous CAC methods on these two benchmarks.