End-to-end learning potentials for structured attribute prediction

TL;DR

This paper introduces a deep neural network approach that incorporates structured inference via conditional random fields to improve multi-attribute prediction accuracy and reveal attribute relationships.

Contribution

It proposes a novel end-to-end trainable framework combining deep neural networks with structured inference using CRFs for attribute prediction.

Findings

Structured inference improves attribute prediction accuracy.

The method uncovers hidden attribute relationships.

Experimental results on SUN and CelebA datasets validate effectiveness.

Abstract

We present a structured inference approach in deep neural networks for multiple attribute prediction. In attribute prediction, a common approach is to learn independent classifiers on top of a good feature representation. However, such classifiers assume conditional independence on features and do not explicitly consider the dependency between attributes in the inference process. We propose to formulate attribute prediction in terms of marginal inference in the conditional random field. We model potential functions by deep neural networks and apply the sum-product algorithm to solve for the approximate marginal distribution in feed-forward networks. Our message passing layer implements sparse pairwise potentials by a softplus-linear function that is equivalent to a higher-order classifier, and learns all the model parameters by end-to-end back propagation. The experimental results using SUN attributes and CelebA datasets suggest that the structured inference improves the attribute prediction performance, and possibly uncovers the hidden relationship between attributes.