Distance for Functional Data Clustering Based on Smoothing Parameter Commutation

TL;DR

This paper introduces a new dissimilarity measure for functional data clustering that uses pairwise smoothing parameter commutation, improving robustness to noise and missing data while maintaining computational efficiency.

Contribution

The novel dissimilarity measure based on pairwise smoothing parameter commutation enhances functional data clustering by handling missing data and outliers effectively.

Findings

Outperforms existing dissimilarity measures in simulations

Handles missing values and irregular time points directly

Maintains computational complexity similar to Euclidean distance

Abstract

We propose a novel method to determine the dissimilarity between subjects for functional data clustering. Spline smoothing or interpolation is common to deal with data of such type. Instead of estimating the best-representing curve for each subject as fixed during clustering, we measure the dissimilarity between subjects based on varying curve estimates with commutation of smoothing parameters pair-by-pair (of subjects). The intuitions are that smoothing parameters of smoothing splines reflect inverse signal-to-noise ratios and that applying an identical smoothing parameter the smoothed curves for two similar subjects are expected to be close. The effectiveness of our proposal is shown through simulations comparing to other dissimilarity measures. It also has several pragmatic advantages. First, missing values or irregular time points can be handled directly, thanks to the nature of smoothing splines. Second, conventional clustering method based on dissimilarity can be employed straightforward, and the dissimilarity also serves as a useful tool for outlier detection. Third, the implementation is almost handy since subroutines for smoothing splines and numerical integration are widely available. Fourth, the computational complexity does not increase and is parallel with that in calculating Euclidean distance between curves estimated by smoothing splines.