Feature Selection and Junta Testing are Statistically Equivalent

TL;DR

This paper proves that junta testing and feature selection are statistically equivalent problems, providing a sample complexity bound that is optimal for both tasks, and demonstrating the effectiveness of a brute-force approach.

Contribution

It establishes the statistical equivalence of junta testing and feature selection, and derives the optimal sample complexity for both tasks.

Findings

Brute-force algorithm is sample-optimal for both problems.

Optimal sample size is rac{1}{\u03b5}(\u221a{2^k inom{n}{k}} + \u221a{inom{n}{k}}).

Junta testing and feature selection are statistically equivalent.

Abstract

For a function $f \colon \{0,1\}^n \to \{0,1\}$, the junta testing problem asks whether $f$ depends on only $k$ variables. If $f$ depends on only $k$ variables, the feature selection problem asks to find those variables. We prove that these two tasks are statistically equivalent. Specifically, we show that the ``brute-force'' algorithm, which checks for any set of $k$ variables consistent with the sample, is simultaneously sample-optimal for both problems, and the optimal sample size is \[ \Theta\left(\frac 1 \varepsilon \left( \sqrt{2^k \log {n \choose k}} + \log {n \choose k}\right)\right). \]