Conditional Linear Regression for Heterogeneous Covariances

TL;DR

This paper introduces a polynomial time algorithm for conditional linear regression that identifies DNF conditions and linear predictors, improving previous methods by handling heterogeneous covariances in data subsets.

Contribution

It presents a novel algorithm for conditional linear regression that relaxes covariance similarity assumptions, enabling better modeling of heterogeneous data.

Findings

Algorithm successfully identifies DNF conditions and linear predictors.

Handles data with diverse covariance structures within conditions.

Improves computational efficiency over prior methods.

Abstract

Often machine learning and statistical models will attempt to describe the majority of the data. However, there may be situations where only a fraction of the data can be fit well by a linear regression model. Here, we are interested in a case where such inliers can be identified by a Disjunctive Normal Form (DNF) formula. We give a polynomial time algorithm for the conditional linear regression task, which identifies a DNF condition together with the linear predictor on the corresponding portion of the data. In this work, we improve on previous algorithms by removing a requirement that the covariances of the data satisfying each of the terms of the condition have to all be very similar in spectral norm to the covariance of the overall condition.