Joint Dimensionality Reduction for Separable Embedding Estimation

TL;DR

This paper introduces a supervised joint dimensionality reduction technique for multi-modal data, providing theoretical guarantees and practical improvements in gene-disease association prediction.

Contribution

It proposes a novel joint linear embedding method with theoretical error bounds and an efficient feature selection process for multi-modal data analysis.

Findings

The method accurately estimates true embeddings under certain conditions.

Numerical experiments validate the theoretical sample complexity results.

The approach outperforms existing methods in gene-disease association prediction.

Abstract

Low-dimensional embeddings for data from disparate sources play critical roles in multi-modal machine learning, multimedia information retrieval, and bioinformatics. In this paper, we propose a supervised dimensionality reduction method that learns linear embeddings jointly for two feature vectors representing data of different modalities or data from distinct types of entities. We also propose an efficient feature selection method that complements, and can be applied prior to, our joint dimensionality reduction method. Assuming that there exist true linear embeddings for these features, our analysis of the error in the learned linear embeddings provides theoretical guarantees that the dimensionality reduction method accurately estimates the true embeddings when certain technical conditions are satisfied and the number of samples is sufficiently large. The derived sample complexity results are echoed by numerical experiments. We apply the proposed dimensionality reduction method to gene-disease association, and predict unknown associations using kernel regression on the dimension-reduced feature vectors. Our approach compares favorably against other dimensionality reduction methods, and against a state-of-the-art method of bilinear regression for predicting gene-disease associations.