From Sets to Multisets: Provable Variational Inference for Probabilistic Integer Submodular Models

TL;DR

This paper introduces a new continuous extension for integer submodular functions, enabling probabilistic modeling and approximate inference, demonstrated on social network datasets.

Contribution

It proposes the Generalized Multilinear Extension for integer submodular functions and develops a block-coordinate ascent algorithm for probabilistic inference.

Findings

Effective inference on real-world social graphs

Demonstrates viability of the new probabilistic model

Shows advantages over existing methods

Abstract

Submodular functions have been studied extensively in machine learning and data mining. In particular, the optimization of submodular functions over the integer lattice (integer submodular functions) has recently attracted much interest, because this domain relates naturally to many practical problem settings, such as multilabel graph cut, budget allocation and revenue maximization with discrete assignments. In contrast, the use of these functions for probabilistic modeling has received surprisingly little attention so far. In this work, we firstly propose the Generalized Multilinear Extension, a continuous DR-submodular extension for integer submodular functions. We study central properties of this extension and formulate a new probabilistic model which is defined through integer submodular functions. Then, we introduce a block-coordinate ascent algorithm to perform approximate inference for those class of models. Finally, we demonstrate its effectiveness and viability on several real-world social connection graph datasets with integer submodular objectives.