Tensor Balancing on Statistical Manifold

TL;DR

This paper introduces an efficient Newton's method-based algorithm for tensor balancing, modeling tensors as probability distributions on a statistical manifold, with broad applications in statistical and machine learning models.

Contribution

The paper presents a novel quadratic convergence algorithm for tensor balancing using Newton's method and a statistical manifold framework, with theoretical proof and practical speed improvements.

Findings

Algorithm is several orders of magnitude faster than existing methods.

Theoretical proof of correctness via manifold projection.

Applicable to various statistical and machine learning models.

Abstract

We solve tensor balancing, rescaling an Nth order nonnegative tensor by multiplying N tensors of order N - 1 so that every fiber sums to one. This generalizes a fundamental process of matrix balancing used to compare matrices in a wide range of applications from biology to economics. We present an efficient balancing algorithm with quadratic convergence using Newton's method and show in numerical experiments that the proposed algorithm is several orders of magnitude faster than existing ones. To theoretically prove the correctness of the algorithm, we model tensors as probability distributions in a statistical manifold and realize tensor balancing as projection onto a submanifold. The key to our algorithm is that the gradient of the manifold, used as a Jacobian matrix in Newton's method, can be analytically obtained using the Moebius inversion formula, the essential of combinatorial mathematics. Our model is not limited to tensor balancing, but has a wide applicability as it includes various statistical and machine learning models such as weighted DAGs and Boltzmann machines.