Temporal Flow Mask Attention for Open-Set Long-Tailed Recognition of Wild Animals in Camera-Trap Images

TL;DR

This paper introduces the Temporal Flow Mask Attention Network, a novel deep learning approach combining optical flow, attention residuals, and meta-embedding techniques to improve open-set long-tailed recognition of wildlife in camera-trap images, especially for unknown classes.

Contribution

The paper presents a new network architecture that integrates optical flow, attention mechanisms, and meta-embedding to address open-set long-tailed recognition in wildlife images.

Findings

Effective in recognizing known and unknown wildlife classes.

Robust to class imbalance and open-set scenarios.

Outperforms existing methods on DMZ dataset.

Abstract

Camera traps, unmanned observation devices, and deep learning-based image recognition systems have greatly reduced human effort in collecting and analyzing wildlife images. However, data collected via above apparatus exhibits 1) long-tailed and 2) open-ended distribution problems. To tackle the open-set long-tailed recognition problem, we propose the Temporal Flow Mask Attention Network that comprises three key building blocks: 1) an optical flow module, 2) an attention residual module, and 3) a meta-embedding classifier. We extract temporal features of sequential frames using the optical flow module and learn informative representation using attention residual blocks. Moreover, we show that applying the meta-embedding technique boosts the performance of the method in open-set long-tailed recognition. We apply this method on a Korean Demilitarized Zone (DMZ) dataset. We conduct extensive experiments, and quantitative and qualitative analyses to prove that our method effectively tackles the open-set long-tailed recognition problem while being robust to unknown classes.