Recognition in Terra Incognita

TL;DR

This paper introduces a new dataset from camera traps to evaluate how well recognition algorithms generalize to unfamiliar environments, revealing current limitations in generalization, especially for classification tasks.

Contribution

It provides a novel dataset for studying recognition generalization across environments and benchmarks existing algorithms, highlighting their poor performance in new locations.

Findings

State-of-the-art algorithms perform well within trained locations.

Recognition generalization to new locations is significantly limited.

Classification systems show especially poor generalization.

Abstract

It is desirable for detection and classification algorithms to generalize to unfamiliar environments, but suitable benchmarks for quantitatively studying this phenomenon are not yet available. We present a dataset designed to measure recognition generalization to novel environments. The images in our dataset are harvested from twenty camera traps deployed to monitor animal populations. Camera traps are fixed at one location, hence the background changes little across images; capture is triggered automatically, hence there is no human bias. The challenge is learning recognition in a handful of locations, and generalizing animal detection and classification to new locations where no training data is available. In our experiments state-of-the-art algorithms show excellent performance when tested at the same location where they were trained. However, we find that generalization to new locations is poor, especially for classification systems.