BVI-RLV: A Fully Registered Dataset and Benchmarks for Low-Light Video Enhancement

TL;DR

This paper introduces a new fully registered low-light video dataset with benchmarks, enabling improved deep learning models for low-light video enhancement by providing high-quality training data and comprehensive evaluation.

Contribution

The authors present a novel, fully registered low-light video dataset with ground truth data and benchmarks across multiple deep learning architectures, addressing data scarcity in LLVE.

Findings

Registered video pairs improve LLVE performance

Models trained on this dataset outperform existing datasets

Benchmarking across CNNs, transformers, diffusion, and state space models

Abstract

Low-light videos often exhibit spatiotemporal incoherent noise, compromising visibility and performance in computer vision applications. One significant challenge in enhancing such content using deep learning is the scarcity of training data. This paper introduces a novel low-light video dataset, consisting of 40 scenes with various motion scenarios under two distinct low-lighting conditions, incorporating genuine noise and temporal artifacts. We provide fully registered ground truth data captured in normal light using a programmable motorized dolly and refine it via an image-based approach for pixel-wise frame alignment across different light levels. We provide benchmarks based on four different technologies: convolutional neural networks, transformers, diffusion models, and state space models (mamba). Our experimental results demonstrate the significance of fully registered video pairs for low-light video enhancement (LLVE) and the comprehensive evaluation shows that the models trained with our dataset outperform those trained with the existing datasets. Our dataset and links to benchmarks are publicly available at https://doi.org/10.21227/mzny-8c77.