Optimal Binary Autoencoding with Pairwise Correlations

TL;DR

This paper introduces a novel binary autoencoder learning method based on pairwise correlations, achieving worst-case optimal reconstruction loss through convex optimization, with competitive experimental results.

Contribution

It formulates binary autoencoding as a biconvex optimization problem that leverages pairwise correlations, resulting in an optimal decoder derived from minimax loss minimization.

Findings

Achieves worst-case optimal reconstruction loss.

Uses a single-layer neural network decoder learned via convex optimization.

Demonstrates efficient binary autoencoding with competitive results.

Abstract

We formulate learning of a binary autoencoder as a biconvex optimization problem which learns from the pairwise correlations between encoded and decoded bits. Among all possible algorithms that use this information, ours finds the autoencoder that reconstructs its inputs with worst-case optimal loss. The optimal decoder is a single layer of artificial neurons, emerging entirely from the minimax loss minimization, and with weights learned by convex optimization. All this is reflected in competitive experimental results, demonstrating that binary autoencoding can be done efficiently by conveying information in pairwise correlations in an optimal fashion.