Few-shot Semantic Encoding and Decoding for Video Surveillance

TL;DR

This paper introduces a novel semantic encoding and decoding approach for surveillance videos that reduces storage and transmission costs using sketch-based compression and few-shot learning, outperforming baseline methods.

Contribution

It presents a new semantic encoding-decoding framework utilizing sketch compression and few-shot training, enhancing efficiency and practicality in surveillance video communication.

Findings

Significantly improved video reconstruction performance.

Effective reduction in storage and transmission with minimal quality loss.

Few-shot training enables quick adaptation to new scenes.

Abstract

With the continuous increase in the number and resolution of video surveillance cameras, the burden of transmitting and storing surveillance video is growing. Traditional communication methods based on Shannon's theory are facing optimization bottlenecks. Semantic communication, as an emerging communication method, is expected to break through this bottleneck and reduce the storage and transmission consumption of video. Existing semantic decoding methods often require many samples to train the neural network for each scene, which is time-consuming and labor-intensive. In this study, a semantic encoding and decoding method for surveillance video is proposed. First, the sketch was extracted as semantic information, and a sketch compression method was proposed to reduce the bit rate of semantic information. Then, an image translation network was proposed to translate the sketch into a video frame with a reference frame. Finally, a few-shot sketch decoding network was proposed to reconstruct video from sketch. Experimental results showed that the proposed method achieved significantly better video reconstruction performance than baseline methods. The sketch compression method could effectively reduce the storage and transmission consumption of semantic information with little compromise on video quality. The proposed method provides a novel semantic encoding and decoding method that only needs a few training samples for each surveillance scene, thus improving the practicality of the semantic communication system.