A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing

TL;DR

This tutorial explains dual decomposition and Lagrangian relaxation techniques for inference in NLP, highlighting their theoretical foundations, practical implementation issues, and broad applicability to complex combinatorial inference problems.

Contribution

It provides a comprehensive overview of Lagrangian relaxation methods, including algorithms, guarantees, and practical considerations, extending their use beyond traditional graphical models.

Findings

Lagrangian relaxation enables inference in complex NLP models.

The tutorial discusses algorithms with formal guarantees.

Practical implementation issues are addressed.

Abstract

Dual decomposition, and more generally Lagrangian relaxation, is a classical method for combinatorial optimization; it has recently been applied to several inference problems in natural language processing (NLP). This tutorial gives an overview of the technique. We describe example algorithms, describe formal guarantees for the method, and describe practical issues in implementing the algorithms. While our examples are predominantly drawn from the NLP literature, the material should be of general relevance to inference problems in machine learning. A central theme of this tutorial is that Lagrangian relaxation is naturally applied in conjunction with a broad class of combinatorial algorithms, allowing inference in models that go significantly beyond previous work on Lagrangian relaxation for inference in graphical models.