# Dimensionality Reduction using Similarity-induced Embeddings

## TL;DR

This paper introduces a novel Dimensionality Reduction framework based on similarity measures, overcoming limitations of traditional methods by modeling target distributions directly through similarities, and demonstrates its effectiveness across various tasks and datasets.

## Contribution

A new Similarity Embedding Framework for Dimensionality Reduction that simplifies optimization and enhances robustness by using similarity matrices instead of second-order statistics.

## Key findings

- Outperforms existing DR techniques on multiple datasets
- Enables fast linear embeddings for complex methods
- Supports supervised and out-of-sample extensions

## Abstract

The vast majority of Dimensionality Reduction (DR) techniques rely on second-order statistics to define their optimization objective. Even though this provides adequate results in most cases, it comes with several shortcomings. The methods require carefully designed regularizers and they are usually prone to outliers. In this work, a new DR framework, that can directly model the target distribution using the notion of similarity instead of distance, is introduced. The proposed framework, called Similarity Embedding Framework, can overcome the aforementioned limitations and provides a conceptually simpler way to express optimization targets similar to existing DR techniques. Deriving a new DR technique using the Similarity Embedding Framework becomes simply a matter of choosing an appropriate target similarity matrix. A variety of classical tasks, such as performing supervised dimensionality reduction and providing out-of-of-sample extensions, as well as, new novel techniques, such as providing fast linear embeddings for complex techniques, are demonstrated in this paper using the proposed framework. Six datasets from a diverse range of domains are used to evaluate the proposed method and it is demonstrated that it can outperform many existing DR techniques.