DCID: Deep Canonical Information Decomposition

TL;DR

This paper introduces DCID, a novel deep learning method for accurately identifying shared information between univariate variables, outperforming existing models in synthetic and real-world MRI data analysis.

Contribution

The paper formalizes univariate shared information retrieval, proposes the DCID model, and introduces ICM as an evaluation metric for ground-truth shared features.

Findings

DCID outperforms baseline models on synthetic data.

ICM effectively quantifies shared feature recovery.

DCID extracts meaningful brain region predictors in MRI data.

Abstract

We consider the problem of identifying the signal shared between two one-dimensional target variables, in the presence of additional multivariate observations. Canonical Correlation Analysis (CCA)-based methods have traditionally been used to identify shared variables, however, they were designed for multivariate targets and only offer trivial solutions for univariate cases. In the context of Multi-Task Learning (MTL), various models were postulated to learn features that are sparse and shared across multiple tasks. However, these methods were typically evaluated by their predictive performance. To the best of our knowledge, no prior studies systematically evaluated models in terms of correctly recovering the shared signal. Here, we formalize the setting of univariate shared information retrieval, and propose ICM, an evaluation metric which can be used in the presence of ground-truth labels, quantifying 3 aspects of the learned shared features. We further propose Deep Canonical Information Decomposition (DCID) - a simple, yet effective approach for learning the shared variables. We benchmark the models on a range of scenarios on synthetic data with known ground-truths and observe DCID outperforming the baselines in a wide range of settings. Finally, we demonstrate a real-life application of DCID on brain Magnetic Resonance Imaging (MRI) data, where we are able to extract more accurate predictors of changes in brain regions and obesity. The code for our experiments as well as the supplementary materials are available at https://github.com/alexrakowski/dcid