Structured Learning via Logistic Regression

TL;DR

This paper introduces a novel approach to structured learning by smoothing inference with entropy, transforming the problem into a series of logistic regression tasks with message-dependent biases.

Contribution

It reveals that smoothed inference simplifies structured learning to logistic regression, enabling extension to broader function classes with an oracle for logistic loss minimization.

Findings

Structured energy functions can be extended beyond linear factors.

The approach simplifies structured learning into logistic regression problems.

It enables the use of complex function classes in structured models.

Abstract

A successful approach to structured learning is to write the learning objective as a joint function of linear parameters and inference messages, and iterate between updates to each. This paper observes that if the inference problem is "smoothed" through the addition of entropy terms, for fixed messages, the learning objective reduces to a traditional (non-structured) logistic regression problem with respect to parameters. In these logistic regression problems, each training example has a bias term determined by the current set of messages. Based on this insight, the structured energy function can be extended from linear factors to any function class where an "oracle" exists to minimize a logistic loss.