Deep Hybrid Model for Region of Interest Detection in Omnidirectional Videos

TL;DR

This paper introduces a hybrid deep learning model designed to accurately detect regions of interest in 360-degree videos, improving streaming efficiency and viewer experience by predicting viewports and enabling intelligent video cuts.

Contribution

The work presents a novel hybrid saliency model specifically tailored for ROI detection in omnidirectional videos, combining preprocessing, deep learning prediction, and post-processing steps.

Findings

The proposed model outperforms existing methods on the 360RAT dataset.

It effectively predicts viewports, reducing head movement during streaming.

The approach enhances streaming efficiency and viewing quality.

Abstract

The main goal of the project is to design a new model that predicts regions of interest in 360$^{\circ}$ videos. The region of interest (ROI) plays an important role in 360$^{\circ}$ video streaming. For example, ROIs are used to predict view-ports, intelligently cut the videos for live streaming, etc so that less bandwidth is used. Detecting view-ports in advance helps reduce the movement of the head while streaming and watching a video via the head-mounted device. Whereas, intelligent cuts of the videos help improve the efficiency of streaming the video to users and enhance the quality of their viewing experience. This report illustrates the secondary task to identify ROIs, in which, we design, train, and test a hybrid saliency model. In this work, we refer to saliency regions to represent the regions of interest. The method includes the processes as follows: preprocessing the video to obtain frames, developing a hybrid saliency model for predicting the region of interest, and finally post-processing the output predictions of the hybrid saliency model to obtain the output region of interest for each frame. Then, we compare the performance of the proposed method with the subjective annotations of the 360RAT dataset.