A Dual Approach for Hierarchical Information-Theoretic Tree Abstractions

TL;DR

This paper establishes a formal connection between two tree-abstraction problems inspired by the information-bottleneck method, introduces algorithms leveraging phase transitions and duality, and validates findings with empirical results.

Contribution

It introduces a dual approach linking hard- and soft-constrained tree abstractions, utilizing Lagrangian duality, phase transitions, and integer programming for improved solutions.

Findings

The dual function relates to the Q-function in Q-tree search.

Tree phase transitions correspond to solutions of the dual problem.

The integer programming relaxation has the integrality property due to total unimodularity.

Abstract

In this paper, we consider establishing a formal connection between two distinct tree-abstraction problems inspired by the information-bottleneck (IB) method. Specifically, we consider the hard- and soft-constrained formulations that have recently appeared in the literature to determine the conditions for which the two approaches are equivalent. Our analysis leverages concepts from Lagrangian relaxation and duality theory to relate the dual function of the hard-constrained problem to the Q-function employed in Q-tree search and shows the connection between tree phase transitions and solutions to the dual problem obtained by exploiting the problem structure. An algorithm is proposed that employs knowledge of the tree phase transitions to find a setting of the dual variable that solves the dual problem. Furthermore, we present an alternative approach to select the dual variable that leverages the integer programming formulation of the hard-constrained problem and the strong duality of linear programming. To obtain a linear program, we establish that a relaxation of the integer programming formulation of the hard-constrained tree-search problem has the integrality property by showing that the program constraint matrix is totally unimodular. Empirical results that corroborate the theoretical developments are presented and discussed throughout.