A Hitchhiker's Guide to Poisson Gradient Estimation

TL;DR

This paper systematically compares Poisson gradient estimation methods, introduces a modified EAT approach with theoretical guarantees, and demonstrates improved performance and robustness in neural modeling tasks.

Contribution

It provides the first comprehensive comparison of EAT and GSM methods, along with a novel EAT modification that ensures unbiased first moments and reduces bias.

Findings

Modified EAT outperforms original methods in distributional fidelity.

EAT-based methods show higher robustness to hyperparameters.

Results are comparable to exact gradient methods in key tasks.

Abstract

Poisson-distributed latent variable models are widely used in computational neuroscience, but differentiating through discrete stochastic samples remains challenging. Two approaches address this: Exponential Arrival Time (EAT) simulation and Gumbel-SoftMax (GSM) relaxation. We provide the first systematic comparison of these methods, along with practical guidance for practitioners. Our main technical contribution is a modification to the EAT method that theoretically guarantees an unbiased first moment (exactly matching the firing rate), and reduces second-moment bias. We evaluate these methods on their distributional fidelity, gradient quality, and performance on two tasks: (1) variational autoencoders with Poisson latents, and (2) partially observable generalized linear models, where latent neural connectivity must be inferred from observed spike trains. Across all metrics, our modified EAT method exhibits better overall performance (often comparable to exact gradients), and substantially higher robustness to hyperparameter choices. Together, our results clarify the trade-offs between these methods and offer concrete recommendations for practitioners working with Poisson latent variable models.