FenceNet: Fine-grained Footwork Recognition in Fencing

TL;DR

FenceNet introduces a skeleton-based neural network approach for fine-grained fencing footwork recognition, achieving high accuracy without wearable sensors and simplifying the analysis process compared to prior methods.

Contribution

The paper presents FenceNet, a novel deep learning architecture that classifies fencing footwork from 2D pose data, outperforming existing methods while eliminating the need for manual feature engineering or wearable sensors.

Findings

FenceNet achieves 85.4% accuracy on Fencing Footwork Dataset.

BiFenceNet surpasses state-of-the-art with 87.6% accuracy.

Method simplifies fencing analysis by using only 2D pose data.

Abstract

Current data analysis for the Canadian Olympic fencing team is primarily done manually by coaches and analysts. Due to the highly repetitive, yet dynamic and subtle movements in fencing, manual data analysis can be inefficient and inaccurate. We propose FenceNet as a novel architecture to automate the classification of fine-grained footwork techniques in fencing. FenceNet takes 2D pose data as input and classifies actions using a skeleton-based action recognition approach that incorporates temporal convolutional networks to capture temporal information. We train and evaluate FenceNet on the Fencing Footwork Dataset (FFD), which contains 10 fencers performing 6 different footwork actions for 10-11 repetitions each (652 total videos). FenceNet achieves 85.4% accuracy under 10-fold cross-validation, where each fencer is left out as the test set. This accuracy is within 1% of the current state-of-the-art method, JLJA (86.3%), which selects and fuses features engineered from skeleton data, depth videos, and inertial measurement units. BiFenceNet, a variant of FenceNet that captures the "bidirectionality" of human movement through two separate networks, achieves 87.6% accuracy, outperforming JLJA. Since neither FenceNet nor BiFenceNet requires data from wearable sensors, unlike JLJA, they could be directly applied to most fencing videos, using 2D pose data as input extracted from off-the-shelf 2D human pose estimators. In comparison to JLJA, our methods are also simpler as they do not require manual feature engineering, selection, or fusion.