Implicit ZCA Whitening Effects of Linear Autoencoders for Recommendation

TL;DR

This paper reveals that linear autoencoders inherently perform ZCA whitening on item features in recommendation systems, enhancing item similarity estimation, especially when applied to low-dimensional embeddings like Item2vec.

Contribution

It establishes a theoretical connection between linear autoencoders and ZCA whitening, demonstrating their effect on item feature vectors in recommendation models.

Findings

Linear autoencoders induce ZCA whitening effects on item features.

Applying autoencoders to low-dimensional embeddings improves similarity estimation.

Preliminary experiments show effectiveness of whitening in recommendation tasks.

Abstract

Recently, in the field of recommendation systems, linear regression (autoencoder) models have been investigated as a way to learn item similarity. In this paper, we show a connection between a linear autoencoder model and ZCA whitening for recommendation data. In particular, we show that the dual form solution of a linear autoencoder model actually has ZCA whitening effects on feature vectors of items, while items are considered as input features in the primal problem of the autoencoder/regression model. We also show the correctness of applying a linear autoencoder to low-dimensional item vectors obtained using embedding methods such as Item2vec to estimate item-item similarities. Our experiments provide preliminary results indicating the effectiveness of whitening low-dimensional item embeddings.