OmniSense: Towards Edge-Assisted Online Analytics for 360-Degree Videos

TL;DR

OmniSense is an edge-assisted framework for real-time 360-degree video analytics that intelligently prunes redundant data and scales analysis models to improve accuracy and efficiency in immersive applications.

Contribution

It introduces a lightweight SRoI prediction algorithm and a resource-aware model scaling approach tailored for 360-degree video analysis.

Findings

Improves accuracy by 19.8% to 114.6% over baselines.

Achieves 2.0x to 2.4x speedups with comparable accuracy.

Demonstrates effectiveness on real-world 360-degree videos.

Abstract

With the reduced hardware costs of omnidirectional cameras and the proliferation of various extended reality applications, more and more $360^\circ$ videos are being captured. To fully unleash their potential, advanced video analytics is expected to extract actionable insights and situational knowledge without blind spots from the videos. In this paper, we present OmniSense, a novel edge-assisted framework for online immersive video analytics. OmniSense achieves both low latency and high accuracy, combating the significant computation and network resource challenges of analyzing $360^\circ$ videos. Motivated by our measurement insights into $360^\circ$ videos, OmniSense introduces a lightweight spherical region of interest (SRoI) prediction algorithm to prune redundant information in $360^\circ$ frames. Incorporating the video content and network dynamics, it then smartly scales vision models to analyze the predicted SRoIs with optimized resource utilization. We implement a prototype of OmniSense with commodity devices and evaluate it on diverse real-world collected $360^\circ$ videos. Extensive evaluation results show that compared to resource-agnostic baselines, it improves the accuracy by $19.8\%$ -- $114.6\%$ with similar end-to-end latencies. Meanwhile, it hits $2.0\times$ -- $2.4\times$ speedups while keeping the accuracy on par with the highest accuracy of baselines.