RRNet: Configurable Real-Time Video Enhancement with Arbitrary Local Lighting Variations

TL;DR

RRNet is a lightweight, configurable real-time video enhancement framework that effectively handles local lighting variations, preserves facial identity, and outperforms previous methods in quality and efficiency for practical applications.

Contribution

Introduces RRNet, a novel, object-aware relighting network with a generative dataset pipeline, enabling real-time, localized lighting adjustments without pixel-aligned training data.

Findings

Outperforms prior methods in low-light enhancement

Supports real-time high-resolution processing

Provides effective localized illumination control

Abstract

With the growing demand for real-time video enhancement in live applications, existing methods often struggle to balance speed and effective exposure control, particularly under uneven lighting. We introduce RRNet (Rendering Relighting Network), a lightweight and configurable framework that achieves a state-of-the-art tradeoff between visual quality and efficiency. By estimating parameters for a minimal set of virtual light sources, RRNet enables localized relighting through a depth-aware rendering module without requiring pixel-aligned training data. This object-aware formulation preserves facial identity and supports real-time, high-resolution performance using a streamlined encoder and lightweight prediction head. To facilitate training, we propose a generative AI-based dataset creation pipeline that synthesizes diverse lighting conditions at low cost. With its interpretable lighting control and efficient architecture, RRNet is well suited for practical applications such as video conferencing, AR-based portrait enhancement, and mobile photography. Experiments show that RRNet consistently outperforms prior methods in low-light enhancement, localized illumination adjustment, and glare removal.