Combinatorial optimization of the coefficient of determination

TL;DR

This paper introduces a novel combinatorial geometry algorithm called quadratic sweep for selecting the optimal subset of points with the highest coefficient of determination, demonstrating high accuracy and efficiency through extensive experiments.

Contribution

The paper presents the quadratic sweep algorithm, a new method leveraging geometric projections and topological sweep to optimize the coefficient of determination in subset selection.

Findings

Achieves optimal subset selection in extensive experiments up to n=30.

Demonstrates the method's efficiency with a topological sweep in (n^5  log n) time.

Provides open-source implementation and reproducible experiments.

Abstract

Robust correlation analysis is among the most critical challenges in statistics. Herein, we develop an efficient algorithm for selecting the $k$- subset of $n$ points in the plane with the highest coefficient of determination $\left( R^2 \right)$. Drawing from combinatorial geometry, we propose a method called the \textit{quadratic sweep} that consists of two steps: (i) projectively lifting the data points into $\mathbb R^5$ and then (ii) iterating over each linearly separable $k$-subset. Its basis is that the optimal set of outliers is separable from its complement in $\mathbb R^2$ by a conic section, which, in $\mathbb R^5$, can be found by a topological sweep in $\Theta \left( n^5 \log n \right)$ time. Although key proofs of quadratic separability remain underway, we develop strong mathematical intuitions for our conjectures, then experimentally demonstrate our method's optimality over several million trials up to $n=30$ without error. Implementations in Julia and fully seeded, reproducible experiments are available at https://github.com/marc-harary/QuadraticSweep.