Estimating Maximally Probable Constrained Relations by Mathematical Programming

TL;DR

This paper introduces a unified probabilistic framework for estimating constrained relations like classification, clustering, and ranking, using convex optimization and linear programming, with practical solutions demonstrated on real data.

Contribution

It develops a family of probability measures that unify various relation estimation problems and provides convex optimization and linear programming solutions, including practical algorithms.

Findings

Convex optimization effectively estimates relation measures.

Linear programming solves relation inference efficiently.

Practical algorithms are validated on real datasets.

Abstract

Estimating a constrained relation is a fundamental problem in machine learning. Special cases are classification (the problem of estimating a map from a set of to-be-classified elements to a set of labels), clustering (the problem of estimating an equivalence relation on a set) and ranking (the problem of estimating a linear order on a set). We contribute a family of probability measures on the set of all relations between two finite, non-empty sets, which offers a joint abstraction of multi-label classification, correlation clustering and ranking by linear ordering. Estimating (learning) a maximally probable measure, given (a training set of) related and unrelated pairs, is a convex optimization problem. Estimating (inferring) a maximally probable relation, given a measure, is a 01-linear program. It is solved in linear time for maps. It is NP-hard for equivalence relations and linear orders. Practical solutions for all three cases are shown in experiments with real data. Finally, estimating a maximally probable measure and relation jointly is posed as a mixed-integer nonlinear program. This formulation suggests a mathematical programming approach to semi-supervised learning.