Information Compression in the AI Era: Recent Advances and Future Challenges

TL;DR

This survey explores recent advances in data compression driven by machine learning, highlighting theoretical developments, deep learning applications, and future challenges in the field.

Contribution

It provides a comprehensive overview of recent progress connecting data compression and machine learning, including new theoretical insights and practical deep learning approaches.

Findings

Deep learning enhances task-based and goal-oriented compression methods.

Theoretical frameworks like rate-distortion-perception guide new research.

Potential of large language models for text compression is discussed.

Abstract

This survey articles focuses on emerging connections between the fields of machine learning and data compression. While fundamental limits of classical (lossy) data compression are established using rate-distortion theory, the connections to machine learning have resulted in new theoretical analysis and application areas. We survey recent works on task-based and goal-oriented compression, the rate-distortion-perception theory and compression for estimation and inference. Deep learning based approaches also provide natural data-driven algorithmic approaches to compression. We survey recent works on applying deep learning techniques to task-based or goal-oriented compression, as well as image and video compression. We also discuss the potential use of large language models for text compression. We finally provide some directions for future research in this promising field.