Time Series Compression using Quaternion Valued Neural Networks and Quaternion Backpropagation

TL;DR

This paper introduces a quaternion neural network-based method for time-series compression that preserves feature relations and improves fault classification accuracy on the Tennessee Eastman Dataset.

Contribution

It develops a novel quaternion neural network with quaternion backpropagation for effective time-series compression and demonstrates superior classification performance over real-valued and baseline methods.

Findings

Outperforms real-valued neural networks in fault classification.

Achieves higher accuracy than baseline downsampling methods.

Improves the SimCLR-TS benchmark from 81.43% to 83.90%.

Abstract

We propose a novel quaternionic time-series compression methodology where we divide a long time-series into segments of data, extract the min, max, mean and standard deviation of these chunks as representative features and encapsulate them in a quaternion, yielding a quaternion valued time-series. This time-series is processed using quaternion valued neural network layers, where we aim to preserve the relation between these features through the usage of the Hamilton product. To train this quaternion neural network, we derive quaternion backpropagation employing the GHR calculus, which is required for a valid product and chain rule in quaternion space. Furthermore, we investigate the connection between the derived update rules and automatic differentiation. We apply our proposed compression method on the Tennessee Eastman Dataset, where we perform fault classification using the compressed data in two settings: a fully supervised one and in a semi supervised, contrastive learning setting. Both times, we were able to outperform real valued counterparts as well as two baseline models: one with the uncompressed time-series as the input and the other with a regular downsampling using the mean. Further, we could improve the classification benchmark set by SimCLR-TS from 81.43% to 83.90%.