Animal Identification with Independent Foreground and Background Modeling

TL;DR

This paper introduces a novel animal identification method that separately models foreground and background features, utilizing advanced segmentation and calibration techniques to improve accuracy and robustness in challenging scenarios.

Contribution

It presents a new approach combining independent foreground and background modeling with Per-Instance Temperature Scaling for calibrated predictions.

Findings

Background prediction error reduced by 8.8% and 22.3% in two tests.

Accuracy doubles in cases of background drift.

Independent modeling improves identification robustness.

Abstract

We propose a method that robustly exploits background and foreground in visual identification of individual animals. Experiments show that their automatic separation, made easy with methods like Segment Anything, together with independent foreground and background-related modeling, improves results. The two predictions are combined in a principled way, thanks to novel Per-Instance Temperature Scaling that helps the classifier to deal with appearance ambiguities in training and to produce calibrated outputs in the inference phase. For identity prediction from the background, we propose novel spatial and temporal models. On two problems, the relative error w.r.t. the baseline was reduced by 22.3% and 8.8%, respectively. For cases where objects appear in new locations, an example of background drift, accuracy doubles.