Simultaneous Estimation of Non-Gaussian Components and their Correlation Structure

TL;DR

This paper introduces a probabilistic model that estimates both non-Gaussian components and their linear and energy correlation structures, enhancing the understanding of dependencies in complex data beyond traditional ICA methods.

Contribution

It proposes a novel probabilistic framework that explicitly models linear and energy correlations among non-Gaussian components, using score matching for simple parameter estimation.

Findings

Improves identifiability of non-Gaussian components with correlation structure

Finds new dependencies in natural image and audio data

Demonstrates effectiveness through simulations and real data applications

Abstract

The statistical dependencies which independent component analysis (ICA) cannot remove often provide rich information beyond the linear independent components. It would thus be very useful to estimate the dependency structure from data. While such models have been proposed, they usually concentrated on higher-order correlations such as energy (square) correlations. Yet, linear correlations are a most fundamental and informative form of dependency in many real data sets. Linear correlations are usually completely removed by ICA and related methods, so they can only be analyzed by developing new methods which explicitly allow for linearly correlated components. In this paper, we propose a probabilistic model of linear non-Gaussian components which are allowed to have both linear and energy correlations. The precision matrix of the linear components is assumed to be randomly generated by a higher-order process and explicitly parametrized by a parameter matrix. The estimation of the parameter matrix is shown to be particularly simple because using score matching, the objective function is a quadratic form. Using simulations with artificial data, we demonstrate that the proposed method improves identifiability of non-Gaussian components by simultaneously learning their correlation structure. Applications on simulated complex cells with natural image input, as well as spectrograms of natural audio data show that the method finds new kinds of dependencies between the components.