Analyzing the Impact of Class Transitions on the Design of Pattern Recognition-based Myoelectric Control Schemes

TL;DR

This paper introduces new metrics to analyze transition errors in pattern recognition-based myoelectric control, revealing that classifiers perform differently during transitions than steady states, which impacts real-world usability.

Contribution

The study proposes novel transition-focused metrics and evaluates classifiers on a dataset with continuous class changes, highlighting the importance of transition performance.

Findings

Linear discriminant classifier outperforms others during transitions and steady states.

Offline accuracy metrics do not predict transition performance.

Transition metrics offer a new perspective for evaluating myoelectric control systems.

Abstract

Despite continued efforts to improve classification accuracy, it has been reported that offline accuracy is a poor indicator of the usability of pattern recognition-based myoelectric control. One potential source of this disparity is the existence of transitions between contraction classes that happen during regular use and are reported to be problematic for pattern recognition systems. Nevertheless, these transitions are often ignored or undefined during both the training and testing processes. In this work, we propose a set of metrics for analyzing the transitions that occur during the voluntary changes between contraction classes during continuous control. These metrics quantify the common types of errors that occur during transitions and compare them to existing metrics that apply only to the steady-state portions of the data. We then use these metrics to analyze transition characteristics of 6 commonly used classifiers on a novel dataset that includes continuous transitions between all combinations of seven different contraction classes. Results show that a linear discriminant classifier consistently outperforms other conventional classifiers during both transitions and steady-state conditions, despite having an almost identical offline performance. Results also show that, although offline training metrics correlate with steady-state performance, they do not correlate with transition performance. These insights suggest that the proposed set of metrics could provide a shift in perspective on the way pattern recognition systems are evaluated and provide a more representative picture of a classifier's performance, potentially narrowing the gap between offline performance and online usability.