Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimization

TL;DR

This survey reviews recent progress in visual signal coding and processing using generative models, covering technologies, standards, and optimization techniques to guide future research and applications.

Contribution

It provides a comprehensive overview of generative models in visual signal coding, including recent advancements, standardization efforts, and optimization methods, which were not extensively covered before.

Findings

Summarizes well-established generative models like VAE, GAN, AR, Normalizing Flows, and Diffusion models.

Highlights recent developments in visual signal restoration, synthesis, and editing.

Discusses fast optimization techniques for practical implementation.

Abstract

This paper provides a survey of the latest developments in visual signal coding and processing with generative models. Specifically, our focus is on presenting the advancement of generative models and their influence on research in the domain of visual signal coding and processing. This survey study begins with a brief introduction of well-established generative models, including the Variational Autoencoder (VAE) models, Generative Adversarial Network (GAN) models, Autoregressive (AR) models, Normalizing Flows and Diffusion models. The subsequent section of the paper explores the advancements in visual signal coding based on generative models, as well as the ongoing international standardization activities. In the realm of visual signal processing, our focus lies on the application and development of various generative models in the research of visual signal restoration. We also present the latest developments in generative visual signal synthesis and editing, along with visual signal quality assessment using generative models and quality assessment for generative models. The practical implementation of these studies is closely linked to the investigation of fast optimization. This paper additionally presents the latest advancements in fast optimization on visual signal coding and processing with generative models. We hope to advance this field by providing researchers and practitioners a comprehensive literature review on the topic of visual signal coding and processing with generative models.