Dim the Lights! -- Low-Rank Prior Temporal Data for Specular-Free Video Recovery

TL;DR

This paper introduces a novel two-step framework for removing specular reflections from videos, leveraging low-rank priors to improve detection and inpainting, especially under complex motion conditions.

Contribution

It presents a new spatially adaptive detection method and a variational low-rank based recovery approach for specular-free video restoration, addressing limitations of previous methods.

Findings

Improved detection accuracy over existing methods

Enhanced inpainting quality in complex motion scenarios

Applicable to object removal tasks

Abstract

The appearance of an object is significantly affected by the illumination conditions in the environment. This is more evident with strong reflective objects as they suffer from more dominant specular reflections, causing information loss and discontinuity in the image domain. In this paper, we present a novel framework for specular-free video recovery with special emphasis on dealing with complex motions coming from objects or camera. Our solution is a twostep approach that allows for both detection and restoration of the damaged regions on video data. We first propose a spatially adaptive detection term that searches for the damage areas. We then introduce a variational solution for specular-free video recovery that allows exploiting spatio-temporal correlations by representing prior data in a low-rank form. We demonstrate that our solution prevents major drawbacks of existing approaches while improving the performance in both detection accuracy and inpainting quality. Finally, we show that our approach can be applied to other problems such as object removal.