Regression from Dependent Observations

TL;DR

This paper develops efficient methods for linear and logistic regression with dependent responses on networks, providing strong consistency results and rates comparable to independent data scenarios.

Contribution

It introduces computationally efficient algorithms with theoretical guarantees for regression models with network-dependent responses, a setting lacking strong prior results.

Findings

Proves strong consistency for coefficient and dependency recovery.

Establishes convergence rates matching independent data cases.

Uses concentration of measure for dependent variables to ensure results.

Abstract

The standard linear and logistic regression models assume that the response variables are independent, but share the same linear relationship to their corresponding vectors of covariates. The assumption that the response variables are independent is, however, too strong. In many applications, these responses are collected on nodes of a network, or some spatial or temporal domain, and are dependent. Examples abound in financial and meteorological applications, and dependencies naturally arise in social networks through peer effects. Regression with dependent responses has thus received a lot of attention in the Statistics and Economics literature, but there are no strong consistency results unless multiple independent samples of the vectors of dependent responses can be collected from these models. We present computationally and statistically efficient methods for linear and logistic regression models when the response variables are dependent on a network. Given one sample from a networked linear or logistic regression model and under mild assumptions, we prove strong consistency results for recovering the vector of coefficients and the strength of the dependencies, recovering the rates of standard regression under independent observations. We use projected gradient descent on the negative log-likelihood, or negative log-pseudolikelihood, and establish their strong convexity and consistency using concentration of measure for dependent random variables.