Toward Real-World Adoption of Portrait Relighting via Hybrid Domain Knowledge Fusion

TL;DR

This paper introduces Hybrid Domain Knowledge Fusion, a method that combines synthetic, OLAT, and real datasets to improve portrait relighting, achieving faster inference and state-of-the-art quality.

Contribution

It proposes a novel fusion paradigm with domain-aware adaptation and knowledge distillation, creating a lightweight model with multi-domain expertise.

Findings

Achieves 6x to 240x inference speedup over previous methods.

Maintains state-of-the-art visual quality in portrait relighting.

Constructs a large synthetic dataset with diverse intrinsics for training.

Abstract

The real-world adoption of portrait relighting is hindered by dataset domain gaps, camera sensitivity, and computational costs. We address these challenges with Hybrid Domain Knowledge Fusion, a paradigm that fuses the specialized strengths of synthetic, One-Light-at-A-Time (OLAT), and real-world datasets into a compact model. Our approach features specialized prior models hardened by domain-aware adaptation, followed by augmented knowledge distillation into a lightweight student model with multi-domain expertise. Our method demonstrates a 6x to 240x inference speedup while maintaining state-of-the-art (SOTA) visual quality in the experiments. Additionally, we construct a massive, high-fidelity synthetic dataset with diverse ground-truth intrinsics to support our training pipeline.