Accuracy and Performance Comparison of Video Action Recognition Approaches

TL;DR

This paper provides a fair comparison of fourteen video action recognition models, evaluating their accuracy and computational performance under consistent training conditions and introducing a new accuracy metric.

Contribution

It offers a comprehensive, standardized comparison of state-of-the-art video action recognition models, including a new accuracy metric and performance analysis on HPC systems.

Findings

Models show varying accuracy levels under consistent training conditions.

Distributed training scales efficiently from 2 to 64 GPUs.

New accuracy metric provides additional insight into model performance.

Abstract

Over the past few years, there has been significant interest in video action recognition systems and models. However, direct comparison of accuracy and computational performance results remain clouded by differing training environments, hardware specifications, hyperparameters, pipelines, and inference methods. This article provides a direct comparison between fourteen off-the-shelf and state-of-the-art models by ensuring consistency in these training characteristics in order to provide readers with a meaningful comparison across different types of video action recognition algorithms. Accuracy of the models is evaluated using standard Top-1 and Top-5 accuracy metrics in addition to a proposed new accuracy metric. Additionally, we compare computational performance of distributed training from two to sixty-four GPUs on a state-of-the-art HPC system.