An Angular-Temporal Interaction Network for Light Field Object Tracking in Low-Light Scenes

TL;DR

This paper introduces a novel light field representation and an angular-temporal interaction network that significantly improve object tracking in low-light scenes by leveraging geometric structural cues and self-supervised learning.

Contribution

It proposes a new light field epipolar-plane structure image (ESI) representation and an angular-temporal interaction network (ATINet) for enhanced low-light object tracking.

Findings

ATINet achieves state-of-the-art performance in single object tracking.

The method effectively extends to multiple object tracking.

The approach improves visual expression and reduces redundancy in light fields.

Abstract

High-quality 4D light field representation with efficient angular feature modeling is crucial for scene perception, as it can provide discriminative spatial-angular cues to identify moving targets. However, recent developments still struggle to deliver reliable angular modeling in the temporal domain, particularly in complex low-light scenes. In this paper, we propose a novel light field epipolar-plane structure image (ESI) representation that explicitly defines the geometric structure within the light field. By capitalizing on the abrupt changes in the angles of light rays within the epipolar plane, this representation can enhance visual expression in low-light scenes and reduce redundancy in high-dimensional light fields. We further propose an angular-temporal interaction network (ATINet) for light field object tracking that learns angular-aware representations from the geometric structural cues and angular-temporal interaction cues of light fields. Furthermore, ATINet can also be optimized in a self-supervised manner to enhance the geometric feature interaction across the temporal domain. Finally, we introduce a large-scale light field low-light dataset for object tracking. Extensive experimentation demonstrates that ATINet achieves state-of-the-art performance in single object tracking. Furthermore, we extend the proposed method to multiple object tracking, which also shows the effectiveness of high-quality light field angular-temporal modeling.