Sketch Interface for Teleoperation of Mobile Manipulator to Enable Intuitive and Intended Operation: A Proof of Concept

TL;DR

This paper presents a web-based sketch interface for mobile manipulators that interprets user sketches to enable intuitive, autonomous operation, reducing workload and improving user experience in complex tasks.

Contribution

It introduces a novel sketch-based interface utilizing machine learning for mobile manipulation, enhancing intuitiveness and accessibility compared to traditional controls.

Findings

Sketch interface reduces user workload in manipulation tasks.

Interface improves intuitiveness over conventional axis control.

Validated with user sketches and comparative grasping experiments.

Abstract

Recent advancements in robotics have underscored the need for effective collaboration between humans and robots. Traditional interfaces often struggle to balance robot autonomy with human oversight, limiting their practical application in complex tasks like mobile manipulation. This study aims to develop an intuitive interface that enables a mobile manipulator to autonomously interpret user-provided sketches, enhancing user experience while minimizing burden. We implemented a web-based application utilizing machine learning algorithms to process sketches, making the interface accessible on mobile devices for use anytime, anywhere, by anyone. In the first validation, we examined natural sketches drawn by users for 27 selected manipulation and navigation tasks, gaining insights into trends related to sketch instructions. The second validation involved comparative experiments with five grasping tasks, showing that the sketch interface reduces workload and enhances intuitiveness compared to conventional axis control interfaces. These findings suggest that the proposed sketch interface improves the efficiency of mobile manipulators and opens new avenues for integrating intuitive human-robot collaboration in various applications.