A Novel Local-Global Feature Fusion Framework for Body-weight Exercise Recognition with Pressure Mapping Sensors

TL;DR

This paper introduces a new framework that combines local and global pressure map features using image processing and YOLO detection to improve body-weight exercise recognition accuracy.

Contribution

It proposes a novel local-global feature fusion method with pressure maps and knowledge distillation, enhancing recognition performance over existing global-only approaches.

Findings

11% improvement in F1 score for exercise recognition

Effective local feature extraction from pressure maps

Knowledge distillation improves global feature preservation

Abstract

We present a novel local-global feature fusion framework for body-weight exercise recognition with floor-based dynamic pressure maps. One step further from the existing studies using deep neural networks mainly focusing on global feature extraction, the proposed framework aims to combine local and global features using image processing techniques and the YOLO object detection to localize pressure profiles from different body parts and consider physical constraints. The proposed local feature extraction method generates two sets of high-level local features consisting of cropped pressure mapping and numerical features such as angular orientation, location on the mat, and pressure area. In addition, we adopt a knowledge distillation for regularization to preserve the knowledge of the global feature extraction and improve the performance of the exercise recognition. Our experimental results demonstrate a notable 11 percent improvement in F1 score for exercise recognition while preserving label-specific features.