WaveFormer: Wavelet Embedding Transformer for Biomedical Signals

TL;DR

WaveFormer is a novel transformer architecture that integrates wavelet decomposition at key stages to effectively capture multi-scale frequency and temporal features in biomedical signals, improving classification performance.

Contribution

It introduces a wavelet-based embedding and positional encoding scheme within transformers, specifically designed for biomedical signals with complex frequency and temporal dynamics.

Findings

Achieves competitive classification accuracy across diverse biomedical datasets.

Effectively captures multi-scale frequency features in long time series.

Demonstrates robustness across varying sequence lengths and channel counts.

Abstract

Biomedical signal classification presents unique challenges due to long sequences, complex temporal dynamics, and multi-scale frequency patterns that are poorly captured by standard transformer architectures. We propose WaveFormer, a transformer architecture that integrates wavelet decomposition at two critical stages: embedding construction, where multi-channel Discrete Wavelet Transform (DWT) extracts frequency features to create tokens containing both time-domain and frequency-domain information, and positional encoding, where Dynamic Wavelet Positional Encoding (DyWPE) adapts position embeddings to signal-specific temporal structure through mono-channel DWT analysis. We evaluate WaveFormer on eight diverse datasets spanning human activity recognition and brain signal analysis, with sequence lengths ranging from 50 to 3000 timesteps and channel counts from 1 to 144. Experimental results demonstrate that WaveFormer achieves competitive performance through comprehensive frequency-aware processing. Our approach provides a principled framework for incorporating frequency-domain knowledge into transformer-based time series classification.