Discussion of: Treelets--An adaptive multi-scale basis for sparse unordered data

TL;DR

The paper discusses the Treelets methodology for multi-scale basis construction in high-dimensional data, highlighting its interpretability and predictive power, while raising questions about its comparison to clustering and PCA, and potential supervised extensions.

Contribution

It provides an analysis of Treelets, emphasizing their advantages in interpretability and prediction, and explores potential improvements through supervised learning and basis sparsity considerations.

Findings

Treelets offer powerful predictions with correlated variables.

Interpretability is enhanced through intuitive variable groupings.

Questions raised about supervised extensions and basis sparsity.

Abstract

We congratulate Lee, Nadler and Wasserman (henceforth LNW) on a very interesting paper on new methodology and supporting theory [arXiv:0707.0481]. Treelets seem to tackle two important problems of modern data analysis at once. For datasets with many variables, treelets give powerful predictions even if variables are highly correlated and redundant. Maybe more importantly, interpretation of the results is intuitive. Useful insights about relevant groups of variables can be gained. Our comments and questions include: (i) Could the success of treelets be replicated by a combination of hierarchical clustering and PCA? (ii) When choosing a suitable basis, treelets seem to be largely an unsupervised method. Could the results be even more interpretable and powerful if treelets would take into account some supervised response variable? (iii) Interpretability of the result hinges on the sparsity of the final basis. Do we expect that the selected groups of variables will always be sufficiently small to be amenable for interpretation?