# Learning Hard Alignments with Variational Inference

## TL;DR

This paper introduces a variational inference approach for training hard attention models, demonstrating improved performance over REINFORCE in phoneme recognition tasks, especially in complex scenarios.

## Contribution

It applies variational inference methods, VIMCO and NVIL, to hard attention training and proposes a novel baseline adapting VIMCO for sequential tasks.

## Key findings

- Outperforms REINFORCE in phoneme recognition accuracy
- More effective in noisy environments
- Shows greater benefits on complex tasks

## Abstract

There has recently been significant interest in hard attention models for tasks such as object recognition, visual captioning and speech recognition. Hard attention can offer benefits over soft attention such as decreased computational cost, but training hard attention models can be difficult because of the discrete latent variables they introduce. Previous work used REINFORCE and Q-learning to approach these issues, but those methods can provide high-variance gradient estimates and be slow to train. In this paper, we tackle the problem of learning hard attention for a sequential task using variational inference methods, specifically the recently introduced VIMCO and NVIL. Furthermore, we propose a novel baseline that adapts VIMCO to this setting. We demonstrate our method on a phoneme recognition task in clean and noisy environments and show that our method outperforms REINFORCE, with the difference being greater for a more complicated task.