Inference in latent factor regression with clusterable features

TL;DR

This paper develops new inferential tools for estimating and making inferences about the regression coefficient in latent factor models with clusterable features, addressing longstanding challenges in variance estimation and model interpretability.

Contribution

It introduces computationally efficient estimators for the regression coefficient without prior knowledge of the number of latent factors or mixture structure, and provides a unified asymptotic distribution analysis.

Findings

Estimator is minimax-rate adaptive.

Provides a closed-form asymptotic variance expression.

Valid for high-dimensional settings with p and K varying with n.

Abstract

Regression models, in which the observed features $X \in \R^p$ and the response $Y \in \R$ depend, jointly, on a lower dimensional, unobserved, latent vector $Z \in \R^K$, with $K< p$, are popular in a large array of applications, and mainly used for predicting a response from correlated features. In contrast, methodology and theory for inference on the regression coefficient $\beta$ relating $Y$ to $Z$ are scarce, since typically the un-observable factor $Z$ is hard to interpret. Furthermore, the determination of the asymptotic variance of an estimator of $\beta$ is a long-standing problem, with solutions known only in a few particular cases. To address some of these outstanding questions, we develop inferential tools for $\beta$ in a class of factor regression models in which the observed features are signed mixtures of the latent factors. The model specifications are practically desirable, in a large array of applications, render interpretability to the components of $Z$, and are sufficient for parameter identifiability. Without assuming that the number of latent factors $K$ or the structure of the mixture is known in advance, we construct computationally efficient estimators of $\beta$, along with estimators of other important model parameters. We benchmark the rate of convergence of $\beta$ by first establishing its $\ell_2$-norm minimax lower bound, and show that our proposed estimator is minimax-rate adaptive. Our main contribution is the provision of a unified analysis of the component-wise Gaussian asymptotic distribution of $\wh \beta$ and, especially, the derivation of a closed form expression of its asymptotic variance, together with consistent variance estimators. The resulting inferential tools can be used when both $K$ and $p$ are independent of the sample size $n$, and when both, or either, $p$ and $K$ vary with $n$, while allowing for $p > n$.