Learning Beam Search Policies via Imitation Learning

TL;DR

This paper introduces a unified imitation learning framework for training beam search policies that explicitly incorporate the beam into the model, providing theoretical guarantees and unifying existing methods.

Contribution

It presents a meta-algorithm for learning beam search policies via imitation learning, unifying and extending previous approaches with new theoretical guarantees.

Findings

The meta-algorithm captures existing learning algorithms.

Provides no-regret guarantees for beam search policy learning.

Unifies various approaches under a single framework.

Abstract

Beam search is widely used for approximate decoding in structured prediction problems. Models often use a beam at test time but ignore its existence at train time, and therefore do not explicitly learn how to use the beam. We develop an unifying meta-algorithm for learning beam search policies using imitation learning. In our setting, the beam is part of the model, and not just an artifact of approximate decoding. Our meta-algorithm captures existing learning algorithms and suggests new ones. It also lets us show novel no-regret guarantees for learning beam search policies.