DUT-LFSaliency: Versatile Dataset and Light Field-to-RGB Saliency Detection

TL;DR

This paper introduces a large-scale light field saliency dataset and a versatile two-stream model that enhances performance and efficiency for saliency detection on both desktop and mobile devices, with effective distillation schemes.

Contribution

The paper presents a comprehensive light field saliency dataset and a novel asymmetrical two-stream model with distillation schemes, improving accuracy and efficiency across platforms.

Findings

State-of-the-art performance of Focal stream.

Top-2 F-measure on DUTLF-V2 with reduced model size.

Significant FPS boost and model compression on mobile devices.

Abstract

Light field data exhibit favorable characteristics conducive to saliency detection. The success of learning-based light field saliency detection is heavily dependent on how a comprehensive dataset can be constructed for higher generalizability of models, how high dimensional light field data can be effectively exploited, and how a flexible model can be designed to achieve versatility for desktop computers and mobile devices. To answer these questions, first we introduce a large-scale dataset to enable versatile applications for RGB, RGB-D and light field saliency detection, containing 102 classes and 4204 samples. Second, we present an asymmetrical two-stream model consisting of the Focal stream and RGB stream. The Focal stream is designed to achieve higher performance on desktop computers and transfer focusness knowledge to the RGB stream, relying on two tailor-made modules. The RGB stream guarantees the flexibility and memory/computation efficiency on mobile devices through three distillation schemes. Experiments demonstrate that our Focal stream achieves state-of-the-arts performance. The RGB stream achieves Top-2 F-measure on DUTLF-V2, which tremendously minimizes the model size by 83% and boosts FPS by 5 times, compared with the best performing method. Furthermore, our proposed distillation schemes are applicable to RGB saliency models, achieving impressive performance gains while ensuring flexibility.