Interactive Removal and Ground Truth for Difficult Shadow Scenes

TL;DR

This paper introduces a user-guided shadow removal method capable of handling complex shadows, along with a validated multi-scene ground truth dataset for benchmarking and comparing shadow removal algorithms.

Contribution

The work presents a novel interactive shadow detection and removal technique and provides the first comprehensive, multi-scene ground truth dataset for shadow removal evaluation.

Findings

Our algorithm outperforms existing methods across various shadow types.

The dataset enables consistent and fair comparison of shadow removal techniques.

The online benchmark facilitates future research and development in shadow removal.

Abstract

A user-centric method for fast, interactive, robust and high-quality shadow removal is presented. Our algorithm can perform detection and removal in a range of difficult cases: such as highly textured and colored shadows. To perform detection an on-the-fly learning approach is adopted guided by two rough user inputs for the pixels of the shadow and the lit area. After detection, shadow removal is performed by registering the penumbra to a normalized frame which allows us efficient estimation of non-uniform shadow illumination changes, resulting in accurate and robust removal. Another major contribution of this work is the first validated and multi-scene category ground truth for shadow removal algorithms. This data set containing 186 images eliminates inconsistencies between shadow and shadow-free images and provides a range of different shadow types such as soft, textured, colored and broken shadow. Using this data, the most thorough comparison of state-of-the-art shadow removal methods to date is performed, showing our proposed new algorithm to outperform the state-of-the-art across several measures and shadow category. To complement our dataset, an online shadow removal benchmark website is also presented to encourage future open comparisons in this challenging field of research.