Saliency-based Multiple Region of Interest Detection from a Single 360{\deg} image

TL;DR

This paper introduces a saliency-based method for detecting multiple Regions of Interest in single 360-degree images, addressing the challenge of information overload by predicting optimal view regions using saliency and data augmentation.

Contribution

It proposes a novel approach combining saliency prediction and data augmentation to effectively identify multiple important regions in 360-degree images.

Findings

The method successfully predicts relevant RoIs that summarize the input image.

Data augmentation improves the robustness of saliency prediction.

Subjective evaluation confirms the effectiveness of the selected regions.

Abstract

360{\deg} images are informative -- it contains omnidirectional visual information around the camera. However, the areas that cover a 360{\deg} image is much larger than the human's field of view, therefore important information in different view directions is easily overlooked. To tackle this issue, we propose a method for predicting the optimal set of Region of Interest (RoI) from a single 360{\deg} image using the visual saliency as a clue. To deal with the scarce, strongly biased training data of existing single 360{\deg} image saliency prediction dataset, we also propose a data augmentation method based on the spherical random data rotation. From the predicted saliency map and redundant candidate regions, we obtain the optimal set of RoIs considering both the saliency within a region and the Interaction-Over-Union (IoU) between regions. We conduct the subjective evaluation to show that the proposed method can select regions that properly summarize the input 360{\deg} image.