Multi-Branch Mutual-Distillation Transformer for EEG-Based Seizure Subtype Classification

TL;DR

This paper introduces the MBMD Transformer, a novel deep learning model that effectively classifies EEG seizure subtypes from small datasets by using multi-branch encoding and mutual knowledge distillation between raw EEG and wavelet features.

Contribution

It presents the first application of knowledge distillation in EEG seizure classification and designs a multi-branch transformer architecture tailored for small data scenarios.

Findings

MBMD Transformer outperforms traditional machine learning methods.

The model achieves superior accuracy on public EEG datasets.

Knowledge transfer between raw EEG and wavelet features enhances classification performance.

Abstract

Cross-subject electroencephalogram (EEG) based seizure subtype classification is very important in precise epilepsy diagnostics. Deep learning is a promising solution, due to its ability to automatically extract latent patterns. However, it usually requires a large amount of training data, which may not always be available in clinical practice. This paper proposes Multi-Branch Mutual-Distillation (MBMD) Transformer for cross-subject EEG-based seizure subtype classification, which can be effectively trained from small labeled data. MBMD Transformer replaces all even-numbered encoder blocks of the vanilla Vision Transformer by our designed multi-branch encoder blocks. A mutual-distillation strategy is proposed to transfer knowledge between the raw EEG data and its wavelets of different frequency bands. Experiments on two public EEG datasets demonstrated that our proposed MBMD Transformer outperformed several traditional machine learning and state-of-the-art deep learning approaches. To our knowledge, this is the first work on knowledge distillation for EEG-based seizure subtype classification.