Learning to Predict Combinatorial Structures

TL;DR

This paper introduces new assumptions and methods for learning algorithms to predict complex combinatorial structures efficiently, addressing computational challenges in structured data prediction.

Contribution

It proposes two novel assumptions—efficiently solvable counting and sampling problems—and develops generalized structured prediction techniques based on them.

Findings

Generalizes ridge regression for structured prediction.

Provides new probabilistic structured prediction models.

Applicable to multi-label, hierarchical classification, and label ranking.

Abstract

The major challenge in designing a discriminative learning algorithm for predicting structured data is to address the computational issues arising from the exponential size of the output space. Existing algorithms make different assumptions to ensure efficient, polynomial time estimation of model parameters. For several combinatorial structures, including cycles, partially ordered sets, permutations and other graph classes, these assumptions do not hold. In this thesis, we address the problem of designing learning algorithms for predicting combinatorial structures by introducing two new assumptions: (i) The first assumption is that a particular counting problem can be solved efficiently. The consequence is a generalisation of the classical ridge regression for structured prediction. (ii) The second assumption is that a particular sampling problem can be solved efficiently. The consequence is a new technique for designing and analysing probabilistic structured prediction models. These results can be applied to solve several complex learning problems including but not limited to multi-label classification, multi-category hierarchical classification, and label ranking.