Vision Beyond Boundaries: An Initial Design Space of Domain-specific Large Vision Models in Human-robot Interaction

TL;DR

This paper introduces a structured design space for domain-specific large vision models tailored for human-robot interaction, emphasizing their potential to improve decision-making and interaction quality.

Contribution

It presents an initial design framework for applying domain-specific large vision models in HRI, filling a research gap and guiding future system development.

Findings

Empirical evaluation with 15 experts across six metrics.

Demonstrated primary efficacy in decision-making scenarios.

Outlined potential application scenarios for HRI.

Abstract

The emergence of large vision models (LVMs) is following in the footsteps of the recent prosperity of Large Language Models (LLMs) in following years. However, there's a noticeable gap in structured research applying LVMs to human-robot interaction (HRI), despite extensive evidence supporting the efficacy of vision models in enhancing interactions between humans and robots. Recognizing the vast and anticipated potential, we introduce an initial design space that incorporates domain-specific LVMs, chosen for their superior performance over normal models. We delve into three primary dimensions: HRI contexts, vision-based tasks, and specific domains. The empirical evaluation was implemented among 15 experts across six evaluated metrics, showcasing the primary efficacy in relevant decision-making scenarios. We explore the process of ideation and potential application scenarios, envisioning this design space as a foundational guideline for future HRI system design, emphasizing accurate domain alignment and model selection.