Human-in-the-loop optimisation: mixed initiative grasping for optimally facilitating post-grasp manipulative actions

TL;DR

This paper presents a mixed initiative system where autonomous agents assist humans in grasping tasks by providing force cues that promote stable and manipulable grasp poses, improving post-grasp manipulation efficiency.

Contribution

The paper introduces a novel haptic shared control system that leverages a task-relevant velocity manipulability cost function to guide human grasp selection for better post-grasp manipulation.

Findings

Grasps minimizing TOV reduce control effort.

Force cues effectively guide human grasp choices.

System improves post-grasp manipulation performance.

Abstract

This paper addresses the problem of mixed initiative, shared control for master-slave grasping and manipulation. We propose a novel system, in which an autonomous agent assists a human in teleoperating a remote slave arm/gripper, using a haptic master device. Our system is designed to exploit the human operator's expertise in selecting stable grasps (still an open research topic in autonomous robotics). Meanwhile, a-priori knowledge of: i) the slave robot kinematics, and ii) the desired post-grasp manipulative trajectory, are fed to an autonomous agent which transmits force cues to the human, to encourage maximally manipulable grasp pose selections. Specifically, the autonomous agent provides force cues to the human, during the reach-to-grasp phase, which encourage the human to select grasp poses which maximise manipulation capability during the post-grasp object manipulation phase. We introduce a task-relevant velocity manipulability cost function (TOV), which is used to identify the maximum kinematic capability of a manipulator during post-grasp motions, and feed this back as force cues to the human during the pre-grasp phase. We show that grasps which minimise TOV result in significantly reduced control effort of the manipulator, compared to other feasible grasps. We demonstrate the effectiveness of our approach by experiments with both real and simulated robots.