Streaming-capable High-performance Architecture of Learned Image Compression Codecs

TL;DR

This paper presents a high-performance architecture for learned image compression codecs that significantly improves runtime efficiency through multi-threaded pipelining and optimized memory models, enabling real-time applications.

Contribution

The authors introduce an architecture-independent method to enhance the runtime performance of learned image codecs, focusing on asynchronous execution and resource utilization.

Findings

Achieves higher throughput and lower latency than baseline implementations.

Enables real-time video streaming with embedded device encoding.

Improves performance without altering neural model structures.

Abstract

Learned image compression allows achieving state-of-the-art accuracy and compression ratios, but their relatively slow runtime performance limits their usage. While previous attempts on optimizing learned image codecs focused more on the neural model and entropy coding, we present an alternative method to improving the runtime performance of various learned image compression models. We introduce multi-threaded pipelining and an optimized memory model to enable GPU and CPU workloads asynchronous execution, fully taking advantage of computational resources. Our architecture alone already produces excellent performance without any change to the neural model itself. We also demonstrate that combining our architecture with previous tweaks to the neural models can further improve runtime performance. We show that our implementations excel in throughput and latency compared to the baseline and demonstrate the performance of our implementations by creating a real-time video streaming encoder-decoder sample application, with the encoder running on an embedded device.