Compression Ratio Learning and Semantic Communications for Video Imaging

TL;DR

This paper introduces a novel video compressed sensing system with spatially-variant ratios and a semantic communication framework using reinforcement learning, improving imaging quality and transmission efficiency for programmable sensors.

Contribution

It presents a new adaptive compression system for video imaging and a semantic communication approach optimized via reinforcement learning, enhancing resource efficiency.

Findings

Higher imaging quality with the proposed system compared to existing methods

Effective trade-off between compression rate and image distortion achieved

Superiority demonstrated over baseline methods in numerical experiments

Abstract

Camera sensors have been widely used in intelligent robotic systems. Developing camera sensors with high sensing efficiency has always been important to reduce the power, memory, and other related resources. Inspired by recent success on programmable sensors and deep optic methods, we design a novel video compressed sensing system with spatially-variant compression ratios, which achieves higher imaging quality than the existing snapshot compressed imaging methods with the same sensing costs. In this article, we also investigate the data transmission methods for programmable sensors, where the performance of communication systems is evaluated by the reconstructed images or videos rather than the transmission of sensor data itself. Usually, different reconstruction algorithms are designed for applications in high dynamic range imaging, video compressive sensing, or motion debluring. This task-aware property inspires a semantic communication framework for programmable sensors. In this work, a policy-gradient based reinforcement learning method is introduced to achieve the explicit trade-off between the compression (or transmission) rate and the image distortion. Numerical results show the superiority of the proposed methods over existing baselines.