Study of the Diffusion Map method in the context of social science data sets -- as an example for spectral dimensionality reduction methods

TL;DR

This paper examines the Diffusion Map method for spectral dimensionality reduction in social science datasets, analyzing its principles, parameters, and effectiveness compared to PCA, with insights on data normalization and variable impact.

Contribution

It provides a comprehensive analysis of the Diffusion Map's fundamental principles, parameter effects, and its application to social science data, highlighting differences from PCA and identifying key influencing factors.

Findings

Time parameter t has negligible effect on results

Data scaling and normalization significantly impact outcomes

Diffusion Map eigenspectrum does not clearly indicate component importance

Abstract

The Diffusion Map is a nonlinear dimensionality reduction technique used to analyze high-dimensional data, with recent applications extending to datasets from the social sciences. Previous research has given little attention to how the specific characteristics of these datasets might influence the results of the Diffusion Map and what conditions must be met for the Diffusion Map to yield meaningful and interpretable results. Moreover, there is a lack of clear, comprehensive explanations of the fundamental principles, which has led to misunderstandings in the literature. This work first addresses the fundamental principles of the Diffusion Map and compares them with other spectral methods. It investigates the impact of the Diffusion Map parameters as well as the structure of the underlying data on the results. The V-Dem democracy dataset, British census data, and data on German urban and rural districts are then analyzed, considering their possible natural parameters. A focus is placed on the benefits of the Diffusion Map in comparison to the established linear principal component analysis (PCA). The analysis shows that the time parameter t of the Diffusion Map framework has no significant influence on the analysis. In contrast, discrete and redundant variables, as well as the scaling and normalization of the data, have a substantial impact. Unlike PCA, the Diffusion Map eigenspectrum does not provide a clear indication of which components are important. Therefore, typical polynomial patterns related to one-dimensional datasets within the Diffusion Map framework are explored. The thesis presents insights suggesting that several underconsidered effects need further examination, and emphasizes the need for a framework to accurately analyze complex datasets using the Diffusion Map.