Maximum Entropy Interval Aggregations

TL;DR

This paper introduces algorithms for optimally and approximately finding contiguous probability aggregations that maximize entropy, with applications in data summarization and information theory.

Contribution

It presents a dynamic programming approach for exact solutions and greedy algorithms for faster, near-optimal solutions to the maximum entropy aggregation problem.

Findings

Dynamic programming computes exact maximum entropy aggregations.

Greedy algorithms offer faster, near-optimal solutions.

The methods are applicable in data compression and probabilistic modeling.

Abstract

Given a probability distribution ${\bf p} = (p_1, \dots, p_n)$ and an integer $1\leq m < n$, we say that ${\bf q} = (q_1, \dots, q_m)$ is a contiguous $m$-aggregation of ${\bf p}$ if there exist indices $0=i_0 < i_1 < \cdots < i_{m-1} < i_m = n$ such that for each $j = 1, \dots, m$ it holds that $q_j = \sum_{k=i_{j-1}+1}^{i_j} p_k.$ In this paper, we consider the problem of efficiently finding the contiguous $m$-aggregation of maximum entropy. We design a dynamic programming algorithm that solves the problem exactly, and two more time-efficient greedy algorithms that provide slightly sub-optimal solutions. We also discuss a few scenarios where our problem matters.