Towards High-Quality Specular Highlight Removal by Leveraging Large-Scale Synthetic Data

TL;DR

This paper introduces a three-stage neural network for removing specular highlights from single object images, leveraging a large synthetic dataset to improve generalization to real-world images with complex lighting.

Contribution

The paper presents a novel three-stage network architecture and a large-scale synthetic dataset for effective specular highlight removal in object images.

Findings

The method generalizes well to real object images.

It produces high-quality, artifact-free specular removal.

The synthetic dataset facilitates training and evaluation.

Abstract

This paper aims to remove specular highlights from a single object-level image. Although previous methods have made some progresses, their performance remains somewhat limited, particularly for real images with complex specular highlights. To this end, we propose a three-stage network to address them. Specifically, given an input image, we first decompose it into the albedo, shading, and specular residue components to estimate a coarse specular-free image. Then, we further refine the coarse result to alleviate its visual artifacts such as color distortion. Finally, we adjust the tone of the refined result to match that of the input as closely as possible. In addition, to facilitate network training and quantitative evaluation, we present a large-scale synthetic dataset of object-level images, covering diverse objects and illumination conditions. Extensive experiments illustrate that our network is able to generalize well to unseen real object-level images, and even produce good results for scene-level images with multiple background objects and complex lighting.