Dictionary-Learning-Based Data Pruning for System Identification

TL;DR

This paper introduces mini-batch FastCan, a dictionary-learning-based data pruning method that effectively reduces sample redundancy in system identification, leading to improved model performance on benchmark datasets.

Contribution

The paper presents a novel sample-wise data pruning technique using dictionary learning, addressing redundancy reduction in system identification.

Findings

Significantly outperforms random pruning.

Effective in reducing sample redundancy.

Improves model coefficient similarity.

Abstract

System identification is normally involved in augmenting time series data by time shifting and nonlinearisation (e.g., polynomial basis), both of which introduce redundancy in features and samples. Many research works focus on reducing redundancy feature-wise, while less attention is paid to sample-wise redundancy. This paper proposes a novel data pruning method, called mini-batch FastCan, to reduce sample-wise redundancy based on dictionary learning. Time series data is represented by some representative samples, called atoms, via dictionary learning. The useful samples are selected based on their correlation with the atoms. The method is tested on one simulated dataset and two benchmark datasets. The R-squared between the coefficients of models trained on the full datasets and the coefficients of models trained on pruned datasets is adopted to evaluate the performance of data pruning methods. It is found that the proposed method significantly outperforms the random pruning method.