Metric Transforms and Low Rank Matrices via Representation Theory of the Real Hyperrectangle

TL;DR

This paper introduces a novel representation theory approach for analyzing matrices related to hyperrectangles, leading to new structural insights in machine learning, metric transforms, and matrix rank properties.

Contribution

It develops a new technique based on representation theory of the real hyperrectangle, providing structural results in kernel functions, metric transforms, and matrix rank behavior.

Findings

Characterization of positive definite Manhattan kernels as completely monotone functions

Complete theory of Manhattan to Manhattan metric transforms via Bernstein functions

Entry-wise polynomial functions on low-rank matrices produce matrices of bounded rank

Abstract

In this paper, we develop a new technique which we call representation theory of the real hyperrectangle, which describes how to compute the eigenvectors and eigenvalues of certain matrices arising from hyperrectangles. We show that these matrices arise naturally when analyzing a number of different algorithmic tasks such as kernel methods, neural network training, natural language processing, and the design of algorithms using the polynomial method. We then use our new technique along with these connections to prove several new structural results in these areas, including: $\bullet$ A function is a positive definite Manhattan kernel if and only if it is a completely monotone function. These kernels are widely used across machine learning; one example is the Laplace kernel which is widely used in machine learning for chemistry. $\bullet$ A function transforms Manhattan distances to Manhattan distances if and only if it is a Bernstein function. This completes the theory of Manhattan to Manhattan metric transforms initiated by Assouad in 1980. $\bullet$ A function applied entry-wise to any square matrix of rank $r$ always results in a matrix of rank $< 2^{r-1}$ if and only if it is a polynomial of sufficiently low degree. This gives a converse to a key lemma used by the polynomial method in algorithm design. Our work includes a sophisticated combination of techniques from different fields, including metric embeddings, the polynomial method, and group representation theory.