Unstructured Human Activity Detection from RGBD Images

TL;DR

This paper presents a hierarchical MEMM-based method for detecting and recognizing unstructured human activities from RGBD data, demonstrating effective performance across various environments and unseen individuals.

Contribution

It introduces a novel hierarchical MEMM approach utilizing RGBD features for activity recognition in unstructured settings, handling unseen subjects.

Findings

Achieved good recognition accuracy across twelve activities.

Performed well in diverse environments like kitchens and offices.

Recognized activities of unseen individuals effectively.

Abstract

Being able to detect and recognize human activities is essential for several applications, including personal assistive robotics. In this paper, we perform detection and recognition of unstructured human activity in unstructured environments. We use a RGBD sensor (Microsoft Kinect) as the input sensor, and compute a set of features based on human pose and motion, as well as based on image and pointcloud information. Our algorithm is based on a hierarchical maximum entropy Markov model (MEMM), which considers a person's activity as composed of a set of sub-activities. We infer the two-layered graph structure using a dynamic programming approach. We test our algorithm on detecting and recognizing twelve different activities performed by four people in different environments, such as a kitchen, a living room, an office, etc., and achieve good performance even when the person was not seen before in the training set.