# SIMCO: SIMilarity-based object COunting

## TL;DR

SIMCO is a novel unsupervised multi-class object counting method that leverages shape and color similarity embeddings for clustering and counting objects in images, achieving state-of-the-art results.

## Contribution

It introduces the first agnostic multi-class object counting approach using a similarity-based embedding and clustering, trained on synthetic data.

## Key findings

- State-of-the-art counting performance on benchmarks
- Effective multi-class unsupervised object counting
- Useful for complex image understanding tasks

## Abstract

We present SIMCO, the first agnostic multi-class object counting approach. SIMCO starts by detecting foreground objects through a novel Mask RCNN-based architecture trained beforehand (just once) on a brand-new synthetic 2D shape dataset, InShape; the idea is to highlight every object resembling a primitive 2D shape (circle, square, rectangle, etc.). Each object detected is described by a low-dimensional embedding, obtained from a novel similarity-based head branch; this latter implements a triplet loss, encouraging similar objects (same 2D shape + color and scale) to map close. Subsequently, SIMCO uses this embedding for clustering, so that different types of objects can emerge and be counted, making SIMCO the very first multi-class unsupervised counter. Experiments show that SIMCO provides state-of-the-art scores on counting benchmarks and that it can also help in many challenging image understanding tasks.