DNN-Driven Compressive Offloading for Edge-Assisted Semantic Video Segmentation

TL;DR

This paper presents STAC, a DNN-driven adaptive compression scheme for edge-assisted semantic video segmentation that significantly reduces bandwidth usage while maintaining high accuracy.

Contribution

STAC introduces a novel DNN-based spatial sensitivity metric for adaptive compression and extends it to videos with a spatiotemporal scheme, reducing bandwidth and preserving segmentation accuracy.

Findings

STAC achieves up to 20.95% bandwidth savings.

It maintains segmentation accuracy comparable to state-of-the-art methods.

The scheme is implemented on a mobile device, demonstrating practical feasibility.

Abstract

Deep learning has shown impressive performance in semantic segmentation, but it is still unaffordable for resource-constrained mobile devices. While offloading computation tasks is promising, the high traffic demands overwhelm the limited bandwidth. Existing compression algorithms are not fit for semantic segmentation, as the lack of obvious and concentrated regions of interest (RoIs) forces the adoption of uniform compression strategies, leading to low compression ratios or accuracy. This paper introduces STAC, a DNN-driven compression scheme tailored for edge-assisted semantic video segmentation. STAC is the first to exploit DNN's gradients as spatial sensitivity metrics for spatial adaptive compression and achieves superior compression ratio and accuracy. Yet, it is challenging to adapt this content-customized compression to videos. Practical issues include varying spatial sensitivity and huge bandwidth consumption for compression strategy feedback and offloading. We tackle these issues through a spatiotemporal adaptive scheme, which (1) takes partial strategy generation operations offline to reduce communication load, and (2) propagates compression strategies and segmentation results across frames through dense optical flow, and adaptively offloads keyframes to accommodate video content. We implement STAC on a commodity mobile device. Experiments show that STAC can save up to 20.95% of bandwidth without losing accuracy, compared to the state-of-the-art algorithm.