A Strongly Polynomial Algorithm for Approximate Forster Transforms and its Application to Halfspace Learning

TL;DR

This paper introduces the first strongly polynomial algorithm for computing approximate Forster transforms, which enables efficient distribution-free PAC learning of halfspaces, even with noise, advancing both geometric data regularization and learning theory.

Contribution

It presents the first strongly polynomial algorithm for approximate Forster transforms and applies it to develop a strongly polynomial halfspace learning algorithm under various noise conditions.

Findings

First strongly polynomial algorithm for approximate Forster transform

Strongly polynomial halfspace learning algorithm in distribution-free setting

Effective learning under random classification noise and Massart noise

Abstract

The Forster transform is a method of regularizing a dataset by placing it in {\em radial isotropic position} while maintaining some of its essential properties. Forster transforms have played a key role in a diverse range of settings spanning computer science and functional analysis. Prior work had given {\em weakly} polynomial time algorithms for computing Forster transforms, when they exist. Our main result is the first {\em strongly polynomial time} algorithm to compute an approximate Forster transform of a given dataset or certify that no such transformation exists. By leveraging our strongly polynomial Forster algorithm, we obtain the first strongly polynomial time algorithm for {\em distribution-free} PAC learning of halfspaces. This learning result is surprising because {\em proper} PAC learning of halfspaces is {\em equivalent} to linear programming. Our learning approach extends to give a strongly polynomial halfspace learner in the presence of random classification noise and, more generally, Massart noise.