Burning sage: Reversing the curse of dimensionality in the visualization of high-dimensional data

TL;DR

This paper introduces the Burning Sage transformation, integrated into the grand tour algorithm, to address the curse of dimensionality in high-dimensional data visualization, enhancing the visibility of data structures.

Contribution

It proposes a novel radial transformation called Burning Sage, incorporated into the grand tour, to improve the visualization of high-dimensional data structures.

Findings

Enhances clustering and classification visualization

Reverses the typical central clumping in projections

Implemented in the R tourr package

Abstract

In high-dimensional data analysis the curse of dimensionality reasons that points tend to be far away from the center of the distribution and on the edge of high-dimensional space. Contrary to this, is that projected data tends to clump at the center. This gives a sense that any structure near the center of the projection is obscured, whether this is true or not. A transformation to reverse the curse, is defined in this paper, which uses radial transformations on the projected data. It is integrated seamlessly into the grand tour algorithm, and we have called it a burning sage tour, to indicate that it reverses the curse. The work is implemented into the tourr package in R. Several case studies are included that show how the sage visualizations enhance exploratory clustering and classification problems.