Explicit Entropic Constructions for Coverage, Facility Location, and Graph Cuts

TL;DR

This paper constructs explicit entropic representations for common submodular functions like coverage, facility location, and graph cuts, bridging combinatorial measures with classical information theory.

Contribution

It provides explicit entropic constructions for key submodular functions, showing they can be realized as Shannon entropies, linking combinatorial and classical information measures.

Findings

Functions can be realized as Shannon entropies of constructed variables

Submodular mutual information aligns with classical mutual information

Conditional gain reduces to conditional entropy

Abstract

Shannon entropy is a polymatroidal set function and lies at the foundation of information theory, yet the class of entropic polymatroids is strictly smaller than the class of all submodular functions. In parallel, submodular and combinatorial information measures (SIMs) have recently been proposed as a principled framework for extending entropy, mutual information, and conditional mutual information to general submodular functions, and have been used extensively in data subset selection, active learning, domain adaptation, and representation learning. This raises a natural and fundamental question: are the monotone submodular functions most commonly used in practice entropic? In this paper, we answer this question in the affirmative for a broad class of widely used polymatroid functions. We provide explicit entropic constructions for set cover and coverage functions, facility location, saturated coverage, concave-over-modular functions via truncations, and monotone graph-cut-type objectives. Our results show that these functions can be realized exactly as Shannon entropies of appropriately constructed random variables. As a consequence, for these functions, submodular mutual information coincides with classical mutual information, conditional gain specializes to conditional entropy, and submodular conditional mutual information reduces to standard conditional mutual information in the entropic sense. These results establish a direct bridge between combinatorial information measures and classical information theory for many of the most common submodular objectives used in applications.