STS Classification with Dual-stream CNN

TL;DR

This paper introduces a dual-stream CNN framework for structured time series classification, effectively modeling complex interwoven spatiotemporal dependencies with high adaptability and modularity.

Contribution

It proposes a novel dual-stream deep neural network inspired by neural science, enhancing flexibility and scalability in modeling interwoven spatiotemporal dependencies.

Findings

Effective on synthetic and benchmark datasets

Outperforms existing methods in accuracy

Highly adaptable to various STS configurations

Abstract

The structured time series (STS) classification problem requires the modeling of interweaved spatiotemporal dependency. most previous STS classification methods model the spatial and temporal dependencies independently. Due to the complexity of the STS data, we argue that a desirable STS classification method should be a holistic framework that can be made as adaptive and flexible as possible. This motivates us to design a deep neural network with such merits. Inspired by the dual-stream hypothesis in neural science, we propose a novel dual-stream framework for modeling the interweaved spatiotemporal dependency, and develop a convolutional neural network within this framework that aims to achieve high adaptability and flexibility in STS configurations from various diagonals, i.e., sequential order, dependency range and features. The proposed architecture is highly modularized and scalable, making it easy to be adapted to specific tasks. The effectiveness of our model is demonstrated through experiments on synthetic data as well as benchmark datasets for skeleton based activity recognition.