A Self-Encoder for Learning Nearest Neighbors

TL;DR

The paper introduces a self-encoder neural network that learns data representations facilitating efficient, scale-invariant nearest neighbor classification, especially effective on heterogeneous datasets with mixed feature types.

Contribution

It proposes a simple self-supervised neural network that learns to embed data for improved nearest neighbor tasks without requiring feature scaling.

Findings

Effective on heterogeneous data with mixed features

Invariant to feature scaling, reducing preprocessing needs

Demonstrates efficiency in various datasets

Abstract

We present the self-encoder, a neural network trained to guess the identity of each data sample. Despite its simplicity, it learns a very useful representation of data, in a self-supervised way. Specifically, the self-encoder learns to distribute the data samples in the embedding space so that they are linearly separable from one another. This induces a geometry where two samples are close in the embedding space when they are not easy to differentiate. The self-encoder can then be combined with a nearest-neighbor classifier or regressor for any subsequent supervised task. Unlike regular nearest neighbors, the predictions resulting from this encoding of data are invariant to any scaling of features, making any preprocessing like min-max scaling not necessary. The experiments show the efficiency of the approach, especially on heterogeneous data mixing numerical features and categorical features.