Curvelet-enhanced transformer architecture for blurred action fine-grained detection

TL;DR

This paper introduces a new network for recognizing human actions in videos, especially under challenging conditions like motion blur.

Contribution

The novel Multi Curvelet Transformer Network (MCTN) uses curvelet transforms to enhance motion clarity and improve action detection.

Findings

MCTN achieves a mean average precision (mAP) of 0.822 on benchmark datasets.

The curvelet-based attention mechanisms improve spatial-temporal feature extraction.

The network shows potential for real-time intelligent video analysis and human-computer interaction.

Abstract

This study proposes a novel Multi Curvelet Transformer Network (MCTN) for fine-grained human behavior recognition in dynamic video scenarios. A key challenge in this field lies in accurately identifying human actions under adverse conditions such as motion blur, occlusion, and varying illumination. To address this, we introduce a motion blur restoration module leveraging the curvelet transform to enhance motion image clarity, thereby improving downstream behavior detection. Furthermore, we enhance the Transformer architecture by embedding curvelet-based multi-scale attention mechanisms, which significantly improve the model’s ability to extract spatial-temporal features at different resolutions. The proposed network also adopts a multi-curvelet transform structure to deepen semantic representation. Experimental results on benchmark datasets, including an action recognition dataset and the MSCOCO dataset, demonstrate that MCTN achieves superior performance, reaching a mean average precision (mAP) of 0.822. These results underscore the potential of MCTN in real-time intelligent video analysis and human-computer interaction applications.

The online version contains supplementary material available at 10.1038/s41598-025-33985-6.

Figures