Meta Learning for High-dimensional Ising Model Selection Using $\ell_1$-regularized Logistic Regression

TL;DR

This paper introduces a meta learning approach for high-dimensional Ising model selection, leveraging auxiliary tasks to reduce sample complexity in neighborhood estimation using $\, ext{l}_1$-regularized logistic regression.

Contribution

It proposes a novel generative model and improper estimation method that effectively pools auxiliary task data to improve support recovery in high-dimensional Ising models.

Findings

Support union recovery with high probability using $\, ext{O}(1)$ samples per task.

Reduced sample complexity for neighborhood selection in the novel task.

Requires $\, ext{O}(d^3 \,\log p)$ tasks for support union estimation.

Abstract

In this paper, we consider the meta learning problem for estimating the graphs associated with high-dimensional Ising models, using the method of $\ell_1$-regularized logistic regression for neighborhood selection of each node. Our goal is to use the information learned from the auxiliary tasks in the learning of the novel task to reduce its sufficient sample complexity. To this end, we propose a novel generative model as well as an improper estimation method. In our setting, all the tasks are \emph{similar} in their \emph{random} model parameters and supports. By pooling all the samples from the auxiliary tasks to \emph{improperly} estimate a single parameter vector, we can recover the true support union, assumed small in size, with a high probability with a sufficient sample complexity of $\Omega(1) $ per task, for $K = \Omega(d^3 \log p ) $ tasks of Ising models with $p$ nodes and a maximum neighborhood size $d$. Then, with the support for the novel task restricted to the estimated support union, we prove that consistent neighborhood selection for the novel task can be obtained with a reduced sufficient sample complexity of $\Omega(d^3 \log d)$.