# Image Representation-Driven Knowledge Distillation for Improved Time-Series Interpretation on Wearable Sensor Data

## TL;DR

This paper explores using image representations in knowledge distillation to improve wearable sensor data interpretation for activity classification.

## Contribution

The novelty lies in investigating how image representations enhance knowledge distillation for better time-series interpretation in wearable systems.

## Key findings

- Image representations integrated into KD improve model performance and compactness in activity classification.
- Different KD strategies using image representations show varied effectiveness across noise, generalizability, and compatibility.
- The study provides insights into balancing representation richness and model efficiency for wearable sensor systems.

## Abstract

With the increased demand for wearable sensors, image representations—such as persistence images and Gramian angular fields—transformed from time-series data have been investigated to address challenges in wearables arising from physiological variations, sensor noise, and limitations in capturing contextual information. To preserve the lightweight structural design of models, knowledge distillation (KD) has also been employed alongside image representations during training to distill smaller and more efficient models. Although image representations play a key role in providing richer and more informative features in training a model, their effectiveness within the KD framework has not been thoroughly explored. In this paper, we focus on image representation-driven KD to investigate whether these representations can provide useful knowledge leading to improved time-series interpretation in activity classification tasks. We explore the benefits of integrating image representations into KD, and we analyze the interplay between representation richness and model compactness with different combinations of teacher and student networks. We also introduce diverse KD strategies to utilize image representations, and we demonstrate the strategies with various perspectives, such as analysis of noises, generalizability, and compatibility, across datasets of varying scales to obtain comprehensive and insightful observations. These offer valuable insights for designing efficient and high-performance wearable sensor-based systems.