# How Compressible are Innovation Processes?

## TL;DR

This paper introduces an entropy-based measure to evaluate the compressibility of continuous-domain innovation processes, comparing stable and Poisson processes, and providing insights into their relative compressibility.

## Contribution

It defines a new entropy limit measure for innovation processes and applies it to compare the compressibility of stable and Poisson processes.

## Key findings

- Poisson innovations are more compressible than stable innovations.
- The entropy measure ranks stable innovations by tail decay.
- The approach extends compressibility analysis to continuous-domain processes.

## Abstract

The sparsity and compressibility of finite-dimensional signals are of great interest in fields such as compressed sensing. The notion of compressibility is also extended to infinite sequences of i.i.d. or ergodic random variables based on the observed error in their nonlinear k-term approximation. In this work, we use the entropy measure to study the compressibility of continuous-domain innovation processes (alternatively known as white noise). Specifically, we define such a measure as the entropy limit of the doubly quantized (time and amplitude) process. This provides a tool to compare the compressibility of various innovation processes. It also allows us to identify an analogue of the concept of "entropy dimension" which was originally defined by R\'enyi for random variables. Particular attention is given to stable and impulsive Poisson innovation processes. Here, our results recognize Poisson innovations as the more compressible ones with an entropy measure far below that of stable innovations. While this result departs from the previous knowledge regarding the compressibility of fat-tailed distributions, our entropy measure ranks stable innovations according to their tail decay.