Kryptonite-N: Machine Learning Strikes Back

TL;DR

This paper challenges the notion that machine learning can universally approximate any function, demonstrating that with specific datasets and polynomial expansion, simple models like logistic regression can achieve universal approximation.

Contribution

It refutes the claim that machine learning cannot universally approximate functions by constructing datasets where simple models succeed with polynomial features.

Findings

Logistic regression with polynomial expansion can approximate any function on Kryptonite-N datasets.

Kryptonite-N datasets are predictably constructed to test universal approximation.

The universal approximation claim for machine learning is not absolute, given specific dataset constructions.

Abstract

Quinn et al propose challenge datasets in their work called ``Kryptonite-N". These datasets aim to counter the universal function approximation argument of machine learning, breaking the notation that machine learning can ``approximate any continuous function" \cite{original_paper}. Our work refutes this claim and shows that universal function approximations can be applied successfully; the Kryptonite datasets are constructed predictably, allowing logistic regression with sufficient polynomial expansion and L1 regularization to solve for any dimension N.