PolarVSR: A Unified Framework and Benchmark for Continuous Space-Time Polarization Video Reconstruction

TL;DR

This paper introduces PolarVSR, a novel framework for continuous space-time polarization video reconstruction, addressing hardware and processing limitations in polarization imaging with a unified neural approach.

Contribution

It presents the first space-time polarization video reconstruction architecture and a large-scale benchmark for this emerging research area.

Findings

The proposed method achieves high-fidelity polarization video reconstruction.

Flow-guided polarization variation loss improves polarization dynamics modeling.

Extensive experiments validate the effectiveness of the approach.

Abstract

Polarimetric imaging captures surface polarization characteristics, such as the Degree of Linear Polarization (DoLP) and the Angle of Polarization (AoP). In mainstream Division of-Focal-Plane (DoFP) color polarization imaging, recovering polarization parameters from captured mosaic arrays remains a challenging inverse problem. Existing DoFP cameras also face hardware bottlenecks and often cannot support high-frame-rate acquisition, limiting polarimetric imaging in dynamic video tasks. These limitations motivate joint spatial and temporal enhancement. To this end, we propose the first space-time polarization video reconstruction architecture. The method jointly models polarization directions in space and time and uses a polarization-aware implicit neural representation for continuous, high-fidelity upsampling. By analyzing temporal variations in polarization parameters, we further introduce a flow-guided polarization variation loss to supervise polarization dynamics. We also establish the first large-scale color DoFP polarization video benchmark to support this research direction. Extensive experiments on this benchmark demonstrate the effectiveness of the method.