# Sparse Quadratic Logistic Regression in Sub-quadratic Time

## TL;DR

This paper introduces a fast, sub-quadratic algorithm for support recovery in sparse quadratic logistic regression, leveraging novel correlation tests and hashing techniques to efficiently identify relevant features.

## Contribution

The paper presents a new algorithm that significantly reduces computation time for support recovery in sparse quadratic logistic regression by combining correlation tests and hashing methods.

## Key findings

- Algorithm achieves sub-quadratic time complexity.
- Effective support recovery demonstrated through experiments.
- Novel correlation test for non-binary finite support cases.

## Abstract

We consider support recovery in the quadratic logistic regression setting - where the target depends on both p linear terms $x_i$ and up to $p^2$ quadratic terms $x_i x_j$. Quadratic terms enable prediction/modeling of higher-order effects between features and the target, but when incorporated naively may involve solving a very large regression problem. We consider the sparse case, where at most $s$ terms (linear or quadratic) are non-zero, and provide a new faster algorithm. It involves (a) identifying the weak support (i.e. all relevant variables) and (b) standard logistic regression optimization only on these chosen variables. The first step relies on a novel insight about correlation tests in the presence of non-linearity, and takes $O(pn)$ time for $n$ samples - giving potentially huge computational gains over the naive approach. Motivated by insights from the boolean case, we propose a non-linear correlation test for non-binary finite support case that involves hashing a variable and then correlating with the output variable. We also provide experimental results to demonstrate the effectiveness of our methods.