On the inconsistency of separable losses for structured prediction

TL;DR

This paper demonstrates that separable negative log-likelihood losses in structured prediction can be inconsistent with the true data distribution, raising concerns about their effectiveness for optimal structured prediction.

Contribution

It provides a theoretical proof that these common losses are not necessarily Bayes consistent, challenging their assumed suitability for structured prediction tasks.

Findings

Separable negative log-likelihood losses may not lead to the most probable structure.

Minimizing these losses does not guarantee Bayes optimal predictions.

The results question the appropriateness of these losses for structured prediction.

Abstract

In this paper, we prove that separable negative log-likelihood losses for structured prediction are not necessarily Bayes consistent, or, in other words, minimizing these losses may not result in a model that predicts the most probable structure in the data distribution for a given input. This fact opens the question of whether these losses are well-adapted for structured prediction and, if so, why.