DeepConvContext: A Multi-Scale Approach to Timeseries Classification in Human Activity Recognition

TL;DR

DeepConvContext introduces a multi-scale framework for human activity recognition that models both intra- and inter-window temporal patterns, significantly improving classification accuracy over traditional window-based methods.

Contribution

It presents a novel multi-scale approach using LSTMs to better capture long-range dependencies in time series data for HAR, outperforming attention-based models.

Findings

Achieves 10% average F1-score improvement over DeepConvLSTM

Demonstrates up to 21% F1-score gains on benchmark datasets

Validates LSTMs outperform attention mechanisms for inertial sensor data

Abstract

Despite recognized limitations in modeling long-range temporal dependencies, Human Activity Recognition (HAR) has traditionally relied on a sliding window approach to segment labeled datasets. Deep learning models like the DeepConvLSTM typically classify each window independently, thereby restricting learnable temporal context to within-window information. To address this constraint, we propose DeepConvContext, a multi-scale time series classification framework for HAR. Drawing inspiration from the vision-based Temporal Action Localization community, DeepConvContext models both intra- and inter-window temporal patterns by processing sequences of time-ordered windows. Unlike recent HAR models that incorporate attention mechanisms, DeepConvContext relies solely on LSTMs -- with ablation studies demonstrating the superior performance of LSTMs over attention-based variants for modeling inertial sensor data. Across six widely-used HAR benchmarks, DeepConvContext achieves an average 10% improvement in F1-score over the classic DeepConvLSTM, with gains of up to 21%. Code to reproduce our experiments is publicly available via github.com/mariusbock/context_har.