Comp-X: On Defining an Interactive Learned Image Compression Paradigm With Expert-driven LLM Agent

TL;DR

Comp-X introduces an interactive image compression paradigm leveraging large language models, unifying multiple coding modes, enabling expert-guided interactions, and establishing a new benchmark for evaluating such systems.

Contribution

It pioneers an interactive, expert-guided image compression framework with a unified coding approach and a dedicated benchmark, advancing towards artificial general intelligence in image coding.

Findings

Efficient understanding of coding requests by LLM agent.

Maintains competitive compression performance across diverse modes.

Demonstrates promising textual interaction capabilities.

Abstract

We present Comp-X, the first intelligently interactive image compression paradigm empowered by the impressive reasoning capability of large language model (LLM) agent. Notably, commonly used image codecs usually suffer from limited coding modes and rely on manual mode selection by engineers, making them unfriendly for unprofessional users. To overcome this, we advance the evolution of image coding paradigm by introducing three key innovations: (i) multi-functional coding framework, which unifies different coding modes of various objective/requirements, including human-machine perception, variable coding, and spatial bit allocation, into one framework. (ii) interactive coding agent, where we propose an augmented in-context learning method with coding expert feedback to teach the LLM agent how to understand the coding request, mode selection, and the use of the coding tools. (iii) IIC-bench, the first dedicated benchmark comprising diverse user requests and the corresponding annotations from coding experts, which is systematically designed for intelligently interactive image compression evaluation. Extensive experimental results demonstrate that our proposed Comp-X can understand the coding requests efficiently and achieve impressive textual interaction capability. Meanwhile, it can maintain comparable compression performance even with a single coding framework, providing a promising avenue for artificial general intelligence (AGI) in image compression.