Classifying Object Manipulation Actions based on Grasp-types and Motion-Constraints

TL;DR

This paper introduces a method for recognizing object manipulation actions by leveraging grasp types and motion constraints, demonstrating improved accuracy over motion-only approaches on the Yale Human Grasping dataset.

Contribution

It proposes using grasp and motion-constraints information for action recognition, providing extensive experimental evaluation and comparison with various classifiers and modeling levels.

Findings

Grasp attributes significantly improve recognition accuracy.

Sequence level modeling outperforms instance level modeling.

Differential attribute analysis highlights key features for classification.

Abstract

In this work, we address a challenging problem of fine-grained and coarse-grained recognition of object manipulation actions. Due to the variations in geometrical and motion constraints, there are different manipulations actions possible to perform different sets of actions with an object. Also, there are subtle movements involved to complete most of object manipulation actions. This makes the task of object manipulation action recognition difficult with only just the motion information. We propose to use grasp and motion-constraints information to recognise and understand action intention with different objects. We also provide an extensive experimental evaluation on the recent Yale Human Grasping dataset consisting of large set of 455 manipulation actions. The evaluation involves a) Different contemporary multi-class classifiers, and binary classifiers with one-vs-one multi- class voting scheme, b) Differential comparisons results based on subsets of attributes involving information of grasp and motion-constraints, c) Fine-grained and Coarse-grained object manipulation action recognition based on fine-grained as well as coarse-grained grasp type information, and d) Comparison between Instance level and Sequence level modeling of object manipulation actions. Our results justifies the efficacy of grasp attributes for the task of fine-grained and coarse-grained object manipulation action recognition.