Hindsight Network Credit Assignment
Kenny Young

TL;DR
Hindsight Network Credit Assignment (HNCA) is a new method for training stochastic neural networks that assigns credit based on immediate child influence, offering unbiased, low-variance gradients with practical computational complexity.
Contribution
HNCA introduces a novel credit assignment technique for stochastic networks, providing unbiased, low-variance gradient estimates with efficiency comparable to backpropagation.
Findings
HNCA outperforms REINFORCE in a MNIST contextual bandit task.
HNCA provides unbiased gradient estimates with reduced variance.
Computational complexity similar to backpropagation.
Abstract
We present Hindsight Network Credit Assignment (HNCA), a novel learning method for stochastic neural networks, which works by assigning credit to each neuron's stochastic output based on how it influences the output of its immediate children in the network. We prove that HNCA provides unbiased gradient estimates while reducing variance compared to the REINFORCE estimator. We also experimentally demonstrate the advantage of HNCA over REINFORCE in a contextual bandit version of MNIST. The computational complexity of HNCA is similar to that of backpropagation. We believe that HNCA can help stimulate new ways of thinking about credit assignment in stochastic compute graphs.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBayesian Modeling and Causal Inference · Data Stream Mining Techniques · Advanced Graph Neural Networks
MethodsREINFORCE
