The Broad Impact of Feature Imitation: Neural Enhancements Across Financial, Speech, and Physiological Domains

TL;DR

This paper demonstrates that Feature Imitating Networks (FINs), which initialize neural weights to approximate statistical features, significantly improve performance across financial, speech, and physiological time series tasks.

Contribution

The study extends the application of FINs beyond biomedical domains to diverse time series datasets, showing their broad utility for performance enhancement.

Findings

Bitcoin prediction error reduced by ~1000 with FINs

Speech emotion recognition accuracy increased by over 3%

Chronic neck pain detection improved by about 7%

Abstract

Initialization of neural network weights plays a pivotal role in determining their performance. Feature Imitating Networks (FINs) offer a novel strategy by initializing weights to approximate specific closed-form statistical features, setting a promising foundation for deep learning architectures. While the applicability of FINs has been chiefly tested in biomedical domains, this study extends its exploration into other time series datasets. Three different experiments are conducted in this study to test the applicability of imitating Tsallis entropy for performance enhancement: Bitcoin price prediction, speech emotion recognition, and chronic neck pain detection. For the Bitcoin price prediction, models embedded with FINs reduced the root mean square error by around 1000 compared to the baseline. In the speech emotion recognition task, the FIN-augmented model increased classification accuracy by over 3 percent. Lastly, in the CNP detection experiment, an improvement of about 7 percent was observed compared to established classifiers. These findings validate the broad utility and potency of FINs in diverse applications.