Scaling Learned Image Compression Models up to 1 Billion

TL;DR

This paper investigates how increasing the size of learned image compression models up to 1 billion parameters affects their performance, revealing scaling laws and achieving state-of-the-art results.

Contribution

It presents the first comprehensive study on scaling learned image compression models, establishing scaling laws and demonstrating improved performance at larger scales.

Findings

Scaling laws relate model size to compression performance.

Scaled-up models achieve state-of-the-art rate-distortion results.

Extrapolation suggests further improvements with larger models.

Abstract

Recent advances in large language models (LLMs) highlight a strong connection between intelligence and compression. Learned image compression, a fundamental task in modern data compression, has made significant progress in recent years. However, current models remain limited in scale, restricting their representation capacity, and how scaling model size influences compression performance remains unexplored. In this work, we present a pioneering study on scaling up learned image compression models and revealing the performance trends through scaling laws. Using the recent state-of-the-art HPCM model as baseline, we scale model parameters from 68.5 millions to 1 billion and fit power-law relations between test loss and key scaling variables, including model size and optimal training compute. The results reveal a scaling trend, enabling extrapolation to larger scale models. Experimental results demonstrate that the scaled-up HPCM-1B model achieves state-of-the-art rate-distortion performance. We hope this work inspires future exploration of large-scale compression models and deeper investigations into the connection between compression and intelligence.