# Notes on Latent Structure Models and SPIGOT

## TL;DR

This paper analyzes the SPIGOT method for training neural networks with discrete latent variables, providing new interpretations, linking it to existing methods, and proposing alternative variants for future exploration.

## Contribution

It offers a novel interpretation of SPIGOT's gradient, connects it to other training methods, and introduces alternative variants for discrete latent variable models.

## Key findings

- Provides a new perspective on SPIGOT's gradient
- Links SPIGOT to other neural network training methods
- Suggests alternative variants for future research

## Abstract

These notes aim to shed light on the recently proposed structured projected intermediate gradient optimization technique (SPIGOT, Peng et al., 2018). SPIGOT is a variant of the straight-through estimator (Bengio et al., 2013) which bypasses gradients of the argmax function by back-propagating a surrogate "gradient." We provide a new interpretation to the proposed gradient and put this technique into perspective, linking it to other methods for training neural networks with discrete latent variables. As a by-product, we suggest alternate variants of SPIGOT which will be further explored in future work.