Star-Graph Multimodal Matching Component Analysis for Data Fusion and Transfer Learning

TL;DR

This paper introduces the star-graph multimodal (SGM) matching component analysis, extending traditional MCA to handle multiple data domains connected via a star-graph, with algorithms and numerical evidence showing improved information encoding.

Contribution

It develops the SGM maps for multimodal data fusion, providing closed-form solutions, algorithms, and a generalized covariance constraint for enhanced data representation.

Findings

SGM encodes more information than MCA with limited training data

Algorithms for SGM map computation and improvement are proposed

Generalized covariance constraint allows larger covariance matrix ranks

Abstract

Previous matching component analysis (MCA) techniques map two data domains to a common domain for further processing in data fusion and transfer learning contexts. In this paper, we extend these techniques to the star-graph multimodal (SGM) case in which one particular data domain is connected to $m$ others via an objective function. We provide a particular feasible point for the resulting trace maximization problem in closed form and algorithms for its computation and iterative improvement, leading to our main result, the SGM maps. We also provide numerical examples demonstrating that SGM is capable of encoding into its maps more information than MCA when few training points are available. In addition, we develop a further generalization of the MCA covariance constraint, eliminating a previous feasibility condition and allowing larger values of the rank of the prescribed covariance matrix.