DeepWiVe: Deep-Learning-Aided Wireless Video Transmission

TL;DR

DeepWiVe introduces an end-to-end deep learning-based wireless video transmission system that integrates compression, channel coding, and modulation, achieving better quality and graceful degradation compared to traditional methods.

Contribution

This work is the first to develop an end-to-end neural network scheme for joint source-channel coding in wireless video transmission, including adaptive bandwidth allocation via reinforcement learning.

Findings

Outperforms H.264 + LDPC in all channel conditions by up to 0.0462 MS-SSIM.

Surpasses H.265 + LDPC by up to 0.0058 MS-SSIM.

Achieves graceful degradation and overcomes the cliff-effect in wireless video transmission.

Abstract

We present DeepWiVe, the first-ever end-to-end joint source-channel coding (JSCC) video transmission scheme that leverages the power of deep neural networks (DNNs) to directly map video signals to channel symbols, combining video compression, channel coding, and modulation steps into a single neural transform. Our DNN decoder predicts residuals without distortion feedback, which improves video quality by accounting for occlusion/disocclusion and camera movements. We simultaneously train different bandwidth allocation networks for the frames to allow variable bandwidth transmission. Then, we train a bandwidth allocation network using reinforcement learning (RL) that optimizes the allocation of limited available channel bandwidth among video frames to maximize overall visual quality. Our results show that DeepWiVe can overcome the cliff-effect, which is prevalent in conventional separation-based digital communication schemes, and achieve graceful degradation with the mismatch between the estimated and actual channel qualities. DeepWiVe outperforms H.264 video compression followed by low-density parity check (LDPC) codes in all channel conditions by up to 0.0462 on average in terms of the multi-scale structural similarity index measure (MS-SSIM), while beating H.265 + LDPC by up to 0.0058 on average. We also illustrate the importance of optimizing bandwidth allocation in JSCC video transmission by showing that our optimal bandwidth allocation policy is superior to the na\"ive uniform allocation. We believe this is an important step towards fulfilling the potential of an end-to-end optimized JSCC wireless video transmission system that is superior to the current separation-based designs.