Light Field Saliency Detection with Deep Convolutional Networks

TL;DR

This paper introduces a new large light field dataset and a novel CNN framework with MAC blocks for improved saliency detection, outperforming existing methods and demonstrating strong generalization.

Contribution

The paper presents a new high-quality light field dataset and a novel CNN architecture with MAC blocks specifically designed for light field saliency detection.

Findings

The new dataset contains 640 light fields with ground-truth saliency maps.

The proposed CNN with MAC blocks outperforms state-of-the-art methods.

The network shows strong generalization on other datasets.

Abstract

Light field imaging presents an attractive alternative to RGB imaging because of the recording of the direction of the incoming light. The detection of salient regions in a light field image benefits from the additional modeling of angular patterns. For RGB imaging, methods using CNNs have achieved excellent results on a range of tasks, including saliency detection. However, it is not trivial to use CNN-based methods for saliency detection on light field images because these methods are not specifically designed for processing light field inputs. In addition, current light field datasets are not sufficiently large to train CNNs. To overcome these issues, we present a new Lytro Illum dataset, which contains 640 light fields and their corresponding ground-truth saliency maps. Compared to current light field saliency datasets [1], [2], our new dataset is larger, of higher quality, contains more variation and more types of light field inputs. This makes our dataset suitable for training deeper networks and benchmarking. Furthermore, we propose a novel end-to-end CNN-based framework for light field saliency detection. Specifically, we propose three novel MAC (Model Angular Changes) blocks to process light field micro-lens images. We systematically study the impact of different architecture variants and compare light field saliency with regular 2D saliency. Our extensive comparisons indicate that our novel network significantly outperforms state-of-the-art methods on the proposed dataset and has desired generalization abilities on other existing datasets.