The Landmark Selection Method for Multiple Output Prediction

TL;DR

This paper introduces a landmark selection method for high-dimensional output prediction, improving over traditional approaches by modeling a subset of output dimensions and then reconstructing the full output, leading to better performance.

Contribution

The paper proposes a novel landmark selection approach for modeling high-dimensional outputs, enhancing prediction accuracy and statistical properties over existing methods.

Findings

Outperforms one-vs-all approach in experiments

Effective in multi-label classification and multivariate regression

Provides statistically advantageous modeling of high-dimensional outputs

Abstract

Conditional modeling x \to y is a central problem in machine learning. A substantial research effort is devoted to such modeling when x is high dimensional. We consider, instead, the case of a high dimensional y, where x is either low dimensional or high dimensional. Our approach is based on selecting a small subset y_L of the dimensions of y, and proceed by modeling (i) x \to y_L and (ii) y_L \to y. Composing these two models, we obtain a conditional model x \to y that possesses convenient statistical properties. Multi-label classification and multivariate regression experiments on several datasets show that this model outperforms the one vs. all approach as well as several sophisticated multiple output prediction methods.