A data-driven functional projection approach for the selection of feature ranges in spectra with ICA or cluster analysis

TL;DR

This paper introduces a data-driven functional projection method for selecting relevant wavelength ranges in spectral data, enhancing prediction accuracy and interpretability in chemometric applications.

Contribution

It proposes a novel functional variable projection approach that adapts to spectral smoothness and employs ICA or clustering for improved feature selection.

Findings

Effective wavelength range selection in infrared spectra

Improved prediction performance on benchmark datasets

Versatile application with ICA and clustering methods

Abstract

Prediction problems from spectra are largely encountered in chemometry. In addition to accurate predictions, it is often needed to extract information about which wavelengths in the spectra contribute in an effective way to the quality of the prediction. This implies to select wavelengths (or wavelength intervals), a problem associated to variable selection. In this paper, it is shown how this problem may be tackled in the specific case of smooth (for example infrared) spectra. The functional character of the spectra (their smoothness) is taken into account through a functional variable projection procedure. Contrarily to standard approaches, the projection is performed on a basis that is driven by the spectra themselves, in order to best fit their characteristics. The methodology is illustrated by two examples of functional projection, using Independent Component Analysis and functional variable clustering, respectively. The performances on two standard infrared spectra benchmarks are illustrated.