Universal Feature Selection with Noisy Observations and Weak Symmetry Conditions

TL;DR

This paper extends universal feature selection methods to noisy data and weak symmetry conditions, demonstrating robustness and near-optimal performance under relaxed assumptions.

Contribution

It introduces weak spherical symmetry and develops a feature selection framework that remains effective despite deviations from ideal symmetry and observation noise.

Findings

Selected features achieve asymptotically optimal error exponents with small symmetry deviations and noise.

The framework is robust to second-moment deviations and noise, broadening practical applicability.

Recovers previous results under ideal symmetry, showing generalization.

Abstract

This paper relaxes the restrictive symmetry conditions adopted in [4], [5] and extends their universal feature selection framework to accommodate noisy observations as well as attribute structures that may exhibit directional preferences. We introduce the notion of weak spherical symmetry, quantified by second-moment distances, which allows controlled deviations from rotational invariance. Under this relaxed condition, we develop a universal feature selection framework based on the singular value decomposition of the canonical dependence matrix computed from noisy data. Our main result shows that the selected features achieve asymptotically optimal error exponents up to a residual term that depends on the symmetry deviation $\delta$ and the noise levels $\eta_1, \eta_2$. When $\delta, \eta_1, \eta_2$ are relatively small, our result recovers that of [5], thereby demonstrating that exact spherical symmetry is unnecessary. Overall, our findings highlight the robustness of the selection framework against second-moment deviations and observation noise, thereby broadening its applicability across diverse inference tasks and providing a theoretically grounded tool for universal feature selection in practical scenarios.