Deep Dependency Networks for Multi-Label Classification

TL;DR

This paper introduces deep dependency networks, combining probabilistic graphical models with deep learning to improve multi-label classification accuracy on image and video datasets.

Contribution

It proposes a novel deep dependency network framework that enhances neural networks with dependency modeling, leading to significant performance improvements.

Findings

Deep dependency networks outperform pure neural architectures.

Significant accuracy gains on multiple video and image datasets.

Effective integration of graphical models with deep learning.

Abstract

We propose a simple approach which combines the strengths of probabilistic graphical models and deep learning architectures for solving the multi-label classification task, focusing specifically on image and video data. First, we show that the performance of previous approaches that combine Markov Random Fields with neural networks can be modestly improved by leveraging more powerful methods such as iterative join graph propagation, integer linear programming, and $\ell_1$ regularization-based structure learning. Then we propose a new modeling framework called deep dependency networks, which augments a dependency network, a model that is easy to train and learns more accurate dependencies but is limited to Gibbs sampling for inference, to the output layer of a neural network. We show that despite its simplicity, jointly learning this new architecture yields significant improvements in performance over the baseline neural network. In particular, our experimental evaluation on three video activity classification datasets: Charades, Textually Annotated Cooking Scenes (TACoS), and Wetlab, and three multi-label image classification datasets: MS-COCO, PASCAL VOC, and NUS-WIDE show that deep dependency networks are almost always superior to pure neural architectures that do not use dependency networks.