Technical Report: Disentangled Action Parsing Networks for Accurate Part-level Action Parsing

TL;DR

This paper introduces Disentangled Action Parsing (DAP), a three-stage model for detailed human action understanding in videos, significantly improving part-level action parsing accuracy.

Contribution

The paper proposes a novel three-stage framework for part-level action parsing, integrating person detection, part parsing, and multi-modal action recognition.

Findings

Achieved a 0.605 mean score in the 2021 Kinetics-TPS Challenge.

Demonstrated effectiveness of disentangled approach in detailed action understanding.

Outperformed existing methods in part-level action parsing accuracy.

Abstract

Part-level Action Parsing aims at part state parsing for boosting action recognition in videos. Despite of dramatic progresses in the area of video classification research, a severe problem faced by the community is that the detailed understanding of human actions is ignored. Our motivation is that parsing human actions needs to build models that focus on the specific problem. We present a simple yet effective approach, named disentangled action parsing (DAP). Specifically, we divided the part-level action parsing into three stages: 1) person detection, where a person detector is adopted to detect all persons from videos as well as performs instance-level action recognition; 2) Part parsing, where a part-parsing model is proposed to recognize human parts from detected person images; and 3) Action parsing, where a multi-modal action parsing network is used to parse action category conditioning on all detection results that are obtained from previous stages. With these three major models applied, our approach of DAP records a global mean of $0.605$ score in 2021 Kinetics-TPS Challenge.