Correlated random features for fast semi-supervised learning

TL;DR

This paper introduces Correlated Nystrom Views (XNV), a fast semi-supervised learning method that uses random features and CCA to improve prediction accuracy and efficiency in regression and classification tasks.

Contribution

XNV combines inexpensive random features with multiview CCA to enhance semi-supervised learning, outperforming existing algorithms in accuracy and speed.

Findings

XNV significantly outperforms state-of-the-art semi-supervised algorithms.

XNV reduces runtime by orders of magnitude.

XNV improves predictive performance across various datasets.

Abstract

This paper presents Correlated Nystrom Views (XNV), a fast semi-supervised algorithm for regression and classification. The algorithm draws on two main ideas. First, it generates two views consisting of computationally inexpensive random features. Second, XNV applies multiview regression using Canonical Correlation Analysis (CCA) on unlabeled data to bias the regression towards useful features. It has been shown that, if the views contains accurate estimators, CCA regression can substantially reduce variance with a minimal increase in bias. Random views are justified by recent theoretical and empirical work showing that regression with random features closely approximates kernel regression, implying that random views can be expected to contain accurate estimators. We show that XNV consistently outperforms a state-of-the-art algorithm for semi-supervised learning: substantially improving predictive performance and reducing the variability of performance on a wide variety of real-world datasets, whilst also reducing runtime by orders of magnitude.