A Novel Unified Architecture for Low-Shot Counting by Detection and Segmentation

TL;DR

GeCo introduces a unified low-shot counting architecture that improves detection, segmentation, and counting accuracy by robust prototype generalization and a novel counting loss, surpassing existing methods.

Contribution

It presents GeCo, a novel architecture with dense object queries and a new counting loss, enhancing low-shot counting performance and accuracy.

Findings

GeCo outperforms previous methods by ~25% in count MAE.

Achieves superior detection accuracy across low-shot setups.

Sets new state-of-the-art results in low-shot counting.

Abstract

Low-shot object counters estimate the number of objects in an image using few or no annotated exemplars. Objects are localized by matching them to prototypes, which are constructed by unsupervised image-wide object appearance aggregation. Due to potentially diverse object appearances, the existing approaches often lead to overgeneralization and false positive detections. Furthermore, the best-performing methods train object localization by a surrogate loss, that predicts a unit Gaussian at each object center. This loss is sensitive to annotation error, hyperparameters and does not directly optimize the detection task, leading to suboptimal counts. We introduce GeCo, a novel low-shot counter that achieves accurate object detection, segmentation, and count estimation in a unified architecture. GeCo robustly generalizes the prototypes across objects appearances through a novel dense object query formulation. In addition, a novel counting loss is proposed, that directly optimizes the detection task and avoids the issues of the standard surrogate loss. GeCo surpasses the leading few-shot detection-based counters by $\sim$25\% in the total count MAE, achieves superior detection accuracy and sets a new solid state-of-the-art result across all low-shot counting setups.