Fast and Accurate Light Field Saliency Detection through Deep Encoding

TL;DR

This paper introduces a novel deep learning approach that significantly speeds up light field saliency detection while maintaining or improving accuracy, by reducing input size and employing attention mechanisms.

Contribution

It proposes a new CNN-based feature extraction and encoding module that enables fast, accurate, and lightweight light field saliency detection using aggressive size reduction.

Findings

Processing time of 0.4 seconds on CPU for large light fields.

Achieves faster performance than state-of-the-art methods.

Maintains or improves accuracy with smaller model size.

Abstract

Light field saliency detection -- important due to utility in many vision tasks -- still lacks speed and can improve in accuracy. Due to the formulation of the saliency detection problem in light fields as a segmentation task or a memorizing task, existing approaches consume unnecessarily large amounts of computational resources for training, and have longer execution times for testing. We solve this by aggressively reducing the large light field images to a much smaller three-channel feature map appropriate for saliency detection using an RGB image saliency detector with attention mechanisms. We achieve this by introducing a novel convolutional neural network based features extraction and encoding module. Our saliency detector takes $0.4$ s to process a light field of size $9\times9\times512\times375$ in a CPU and is significantly faster than state-of-the-art light field saliency detectors, with better or comparable accuracy. Furthermore, model size of our architecture is significantly lower compared to state-of-the-art light field saliency detectors. Our work shows that extracting features from light fields through aggressive size reduction and the attention mechanism results in a faster and accurate light field saliency detector leading to near real-time light field processing.