Learning Sequence Neighbourhood Metrics

TL;DR

This paper introduces a method combining RNNs, pooling, and NCA to learn meaningful sequence embeddings, enabling efficient visualization and classification in fixed-length vector spaces.

Contribution

It presents a novel approach for metric learning on sequential data using RNNs and NCA, facilitating linear-time classification and visualization.

Findings

Sequences are effectively embedded into fixed-length vectors.

Embeddings are meaningful for visualization and nearest neighbor classification.

Method enables linear-time operations on sequential data.

Abstract

Recurrent neural networks (RNNs) in combination with a pooling operator and the neighbourhood components analysis (NCA) objective function are able to detect the characterizing dynamics of sequences and embed them into a fixed-length vector space of arbitrary dimensionality. Subsequently, the resulting features are meaningful and can be used for visualization or nearest neighbour classification in linear time. This kind of metric learning for sequential data enables the use of algorithms tailored towards fixed length vector spaces such as R^n.