Kinematic-Layout-aware Random Forests for Depth-based Action Recognition

TL;DR

This paper introduces a novel kinematic-layout-aware random forest method for depth-based action recognition, effectively integrating scene layout and skeleton information during training to improve accuracy in patient monitoring scenarios.

Contribution

The proposed method uniquely incorporates kinematic-layout information into random forests during training, without requiring it at test time, enhancing action recognition accuracy.

Findings

Outperforms state-of-the-art methods on multiple datasets

Effective in cross-view and view-invariant scenarios

Improves recognition of subtle and significant movements

Abstract

In this paper, we tackle the problem of 24 hours-monitoring patient actions in a ward such as "stretching an arm out of the bed", "falling out of the bed", where temporal movements are subtle or significant. In the concerned scenarios, the relations between scene layouts and body kinematics (skeletons) become important cues to recognize actions; however they are hard to be secured at a testing stage. To address this problem, we propose a kinematic-layout-aware random forest which takes into account the kinematic-layout (\ie layout and skeletons), to maximize the discriminative power of depth image appearance. We integrate the kinematic-layout in the split criteria of random forests to guide the learning process by 1) determining the switch to either the depth appearance or the kinematic-layout information, and 2) implicitly closing the gap between two distributions obtained by the kinematic-layout and the appearance, when the kinematic-layout appears useful. The kinematic-layout information is not required for the test data, thus called "privileged information prior". The proposed method has also been testified in cross-view settings, by the use of view-invariant features and enforcing the consistency among synthetic-view data. Experimental evaluations on our new dataset PATIENT, CAD-60 and UWA3D (multiview) demonstrate that our method outperforms various state-of-the-arts.