EBuddy: a workflow orchestrator for industrial human-machine collaboration

TL;DR

EBuddy is a voice-guided workflow orchestrator that enhances industrial human-machine collaboration by translating expert procedures into interpretable, state-driven systems, reducing process time and operator burden.

Contribution

The paper introduces EBuddy, a novel voice-based system that operationalizes expert workflows as finite state machines for scalable, interpretable industrial collaboration.

Findings

Significant reduction in process duration for impeller blade inspection and repair.

Maintains repeatability and low operator burden during complex tasks.

Successfully integrates heterogeneous resources including robots and software.

Abstract

This paper presents EBuddy, a voice-guided workflow orchestrator for natural human-machine collaboration in industrial environments. EBuddy targets a recurrent bottleneck in tool-intensive workflows: expert know-how is effective but difficult to scale, and execution quality degrades when procedures are reconstructed ad hoc across operators and sessions. EBuddy operationalizes expert practice as a finite state machine (FSM) driven application that provides an interpretable decision frame at runtime (current state and admissible actions), so that spoken requests are interpreted within state-grounded constraints, while the system executes and monitors the corresponding tool interactions. Through modular workflow artifacts, EBuddy coordinates heterogeneous resources, including GUI-driven software and a collaborative robot, leveraging fully voice-based interaction through automatic speech recognition and intent understanding. An industrial pilot on impeller blade inspection and repair preparation for directed energy deposition (DED), realized by human-robot collaboration, shows substantial reductions in end-to-end process duration across onboarding, 3D scanning and processing, and repair program generation, while preserving repeatability and low operator burden.