VC-FeS: Viewpoint-Conditioned Feature Selection for Vehicle Re-identification in Thermal Vision

TL;DR

This paper introduces VC-FeS, a viewpoint-conditioned feature selection method that significantly improves vehicle re-identification accuracy in thermal images by leveraging and adapting RGB pre-trained ViT features.

Contribution

The paper proposes viewpoint-conditioned feature vectors and area-specific comparisons to enhance thermal vehicle re-identification, effectively adapting RGB-trained models to thermal data.

Findings

Achieved 19.7% and 12.8% improvements in mAP on two datasets.

Surpassed state-of-the-art methods in thermal vehicle re-identification.

Plan to release a new thermal maritime vessel dataset.

Abstract

Identification of less-articulated objects using single-channel images, such as thermal images, is important in many applications, such as surveillance. However, in this domain, existing methods show poor performance due to high similarity among objects of the same category in the absence of color information (overlooking shape information) and de-emphasized texture information. Furthermore, variability in viewpoint adds more complexity as the features vary from side to side. We address these issues by constructing viewpoint-conditioned feature vectors and area-specific feature comparisons in separate feature spaces. These interventions enable leveraging the advancements of existing RGB-pre-trained ViT feature extractors while effectively adapting them to address the challenges specific to the thermal domain. We test our system with RGBNT100 (IR) vehicle dataset and a thermal maritime dataset acquired by us. Our results surpass the state-of-the-art methods by 19.7% and 12.8% for the above datasets in mAP scores, respectively. We also plan to make our thermal dataset available, the first of its kind for maritime vessel identification.