Weakly Supervised Face and Whole Body Recognition in Turbulent Environments

TL;DR

This paper introduces a weakly supervised framework for face and person recognition in turbulent environments, improving accuracy without needing paired images or extensive annotations, by aligning turbulent and pristine images in a common space.

Contribution

The paper proposes a novel weakly supervised approach with a self-attention module and tilt map estimator to enhance recognition in turbulent conditions without relying on paired datasets.

Findings

Up to 13.86% improvement in rank-1 accuracy.

Effective recognition under varying turbulence and distances.

Requires fewer annotated samples for training.

Abstract

Face and person recognition have recently achieved remarkable success under challenging scenarios, such as off-pose and cross-spectrum matching. However, long-range recognition systems are often hindered by atmospheric turbulence, leading to spatially and temporally varying distortions in the image. Current solutions rely on generative models to reconstruct a turbulent-free image, but often preserve photo-realism instead of discriminative features that are essential for recognition. This can be attributed to the lack of large-scale datasets of turbulent and pristine paired images, necessary for optimal reconstruction. To address this issue, we propose a new weakly supervised framework that employs a parameter-efficient self-attention module to generate domain agnostic representations, aligning turbulent and pristine images into a common subspace. Additionally, we introduce a new tilt map estimator that predicts geometric distortions observed in turbulent images. This estimate is used to re-rank gallery matches, resulting in up to 13.86\% improvement in rank-1 accuracy. Our method does not require synthesizing turbulent-free images or ground-truth paired images, and requires significantly fewer annotated samples, enabling more practical and rapid utility of increasingly large datasets. We analyze our framework using two datasets -- Long-Range Face Identification Dataset (LRFID) and BRIAR Government Collection 1 (BGC1) -- achieving enhanced discriminability under varying turbulence and standoff distance.