Interactive Robot Programming for Surface Finishing via Task-Centric Mixed Reality Interfaces

TL;DR

This paper introduces an intuitive, task-focused mixed reality interface for programming robots to perform surface finishing tasks, reducing setup complexity and enabling non-experts to effectively deploy robots in small-scale manufacturing.

Contribution

It presents a novel interactive programming approach with a surface segmentation algorithm and optimized user interface, facilitating robot programming by non-experts in surface finishing tasks.

Findings

Reduced user workload in robot programming

Enhanced usability for non-expert users

Effective task programming demonstrated in user studies

Abstract

Lengthy setup processes that require robotics expertise remain a major barrier to deploying robots for tasks involving high product variability and small batch sizes. As a result, collaborative robots, despite their advanced sensing and control capabilities, are rarely used for surface finishing in small-scale craft and manufacturing settings. To address this gap, we propose a novel robot programming approach that enables non-experts to intuitively program robots through interactive, task-focused workflows. For that, we developed a new surface segmentation algorithm that incorporates human input to identify and refine workpiece regions for processing. Throughout the programming process, users receive continuous visual feedback on the robot's learned model, enabling them to iteratively refine the segmentation result. Based on the segmented surface model, a robot trajectory is generated to cover the desired processing area. We evaluated multiple interaction designs across two comprehensive user studies to derive an optimal interface that significantly reduces user workload, improves usability and enables effective task programming even for users with limited practical experience.