Designing Interaction for Multi-agent Cooperative System in an Office Environment

TL;DR

This paper presents the design and implementation of a multi-modal human-machine interface for an intelligent cyber-physical system coordinating multiple agents in an office environment, enhancing user interaction and task efficiency.

Contribution

It introduces a novel multi-model interaction approach for multi-agent systems in office environments, integrating graphical, speech, gesture, and facial expression inputs.

Findings

Effective multi-modal interaction improves user engagement.

System successfully coordinates diverse agents to perform tasks efficiently.

Enhanced supervision and feedback mechanisms are achieved.

Abstract

Future intelligent system will involve very various types of artificial agents, such as mobile robots, smart home infrastructure or personal devices, which share data and collaborate with each other to execute certain tasks.Designing an efficient human-machine interface, which can support users to express needs to the system, supervise the collaboration progress of different entities and evaluate the result, will be challengeable. This paper presents the design and implementation of the human-machine interface of Intelligent Cyber-Physical system (ICPS),which is a multi-entity coordination system of robots and other smart devices in a working environment. ICPS gathers sensory data from entities and then receives users' command, then optimizes plans to utilize the capability of different entities to serve people. Using multi-model interaction methods, e.g. graphical interfaces, speech interaction, gestures and facial expressions, ICPS is able to receive inputs from users through different entities, keep users aware of the progress and accomplish the task efficiently