Transformer-based dimensionality reduction

TL;DR

This paper introduces Transformer-DR, a novel dimensionality reduction method based on Vision Transformer, demonstrating its effectiveness in data visualization, image reconstruction, and face recognition compared to existing methods.

Contribution

It proposes a new Transformer-based DR model, Transformer-DR, leveraging Vision Transformer architecture for improved data representation.

Findings

Transformer-DR outperforms traditional DR methods in visualization tasks

Transformer-DR achieves better image reconstruction quality

Transformer-DR enhances face recognition accuracy

Abstract

Recently, Transformer is much popular and plays an important role in the fields of Machine Learning (ML), Natural Language Processing (NLP), and Computer Vision (CV), etc. In this paper, based on the Vision Transformer (ViT) model, a new dimensionality reduction (DR) model is proposed, named Transformer-DR. From data visualization, image reconstruction and face recognition, the representation ability of Transformer-DR after dimensionality reduction is studied, and it is compared with some representative DR methods to understand the difference between Transformer-DR and existing DR methods. The experimental results show that Transformer-DR is an effective dimensionality reduction method.