# A Neural Network model with Bidirectional Whitening

## TL;DR

This paper introduces a bidirectional whitening neural network model that enhances natural gradient descent in multilayer perceptrons by applying whitening during both feed-forward and back-propagation phases, improving learning efficiency.

## Contribution

It extends whitened neural networks by incorporating whitening in both directions, offering a novel approach to natural gradient descent in neural networks.

## Key findings

- Effective on MNIST handwritten character recognition
- Improves training efficiency of multilayer perceptrons
- Demonstrates the benefits of bidirectional whitening

## Abstract

We present here a new model and algorithm which performs an efficient Natural gradient descent for Multilayer Perceptrons. Natural gradient descent was originally proposed from a point of view of information geometry, and it performs the steepest descent updates on manifolds in a Riemannian space. In particular, we extend an approach taken by the "Whitened neural networks" model. We make the whitening process not only in feed-forward direction as in the original model, but also in the back-propagation phase. Its efficacy is shown by an application of this "Bidirectional whitened neural networks" model to a handwritten character recognition data (MNIST data).