Image-to-Image Translation of Synthetic Samples for Rare Classes

TL;DR

This paper investigates using image-to-image translation to improve the realism of synthetic images for rare species classification, reducing domain gap and enhancing classifier performance in wildlife monitoring.

Contribution

It introduces a method to align low-level features between synthetic and real images, improving synthetic data quality for rare class classification.

Findings

Significant reduction in classification error rates for rare species.

Effective domain gap closure between synthetic and real images.

Improved classifier performance with aligned synthetic data.

Abstract

The natural world is long-tailed: rare classes are observed orders of magnitudes less frequently than common ones, leading to highly-imbalanced data where rare classes can have only handfuls of examples. Learning from few examples is a known challenge for deep learning based classification algorithms, and is the focus of the field of low-shot learning. One potential approach to increase the training data for these rare classes is to augment the limited real data with synthetic samples. This has been shown to help, but the domain shift between real and synthetic hinders the approaches' efficacy when tested on real data. We explore the use of image-to-image translation methods to close the domain gap between synthetic and real imagery for animal species classification in data collected from camera traps: motion-activated static cameras used to monitor wildlife. We use low-level feature alignment between source and target domains to make synthetic data for a rare species generated using a graphics engine more "realistic". Compared against a system augmented with unaligned synthetic data, our experiments show a considerable decrease in classification error rates on a rare species.