Towards Language-Based Modulation of Assistive Robots through Multimodal Models

TL;DR

This research explores using multimodal language cues to enable assistive robots to adapt and respond in real-time, aiming to improve human-robot communication and interaction in Geriatronics.

Contribution

It introduces a novel approach for real-time, language-based modulation of assistive robots using multimodal inputs, emphasizing transparency and interpretability.

Findings

Proposed a framework for multimodal language-based modulation

Emphasized transparency and interpretability in robot responses

Aimed to enhance usability and flexibility of assistive robots

Abstract

In the field of Geriatronics, enabling effective and transparent communication between humans and robots is crucial for enhancing the acceptance and performance of assistive robots. Our early-stage research project investigates the potential of language-based modulation as a means to improve human-robot interaction. We propose to explore real-time modulation during task execution, leveraging language cues, visual references, and multimodal inputs. By developing transparent and interpretable methods, we aim to enable robots to adapt and respond to language commands, enhancing their usability and flexibility. Through the exchange of insights and knowledge at the workshop, we seek to gather valuable feedback to advance our research and contribute to the development of interactive robotic systems for Geriatronics and beyond.