To Ask or Not To Ask: Human-in-the-loop Contextual Bandits with Applications in Robot-Assisted Feeding

TL;DR

This paper introduces LinUCB-QG, a human-in-the-loop contextual bandit method for robot-assisted feeding that intelligently balances task success and user workload by selectively querying feedback, validated through simulations and user studies.

Contribution

It presents a novel framework combining contextual bandits with workload prediction to optimize human-robot interaction in food acquisition tasks.

Findings

Better balance between task success and user workload compared to baselines.

Adaptive querying reduces workload for users with mobility limitations.

Effective in both simulated and real user scenarios.

Abstract

Robot-assisted bite acquisition involves picking up food items with varying shapes, compliance, sizes, and textures. Fully autonomous strategies may not generalize efficiently across this diversity. We propose leveraging feedback from the care recipient when encountering novel food items. However, frequent queries impose a workload on the user. We formulate human-in-the-loop bite acquisition within a contextual bandit framework and introduce LinUCB-QG, a method that selectively asks for help using a predictive model of querying workload based on query types and timings. This model is trained on data collected in an online study involving 14 participants with mobility limitations, 3 occupational therapists simulating physical limitations, and 89 participants without limitations. We demonstrate that our method better balances task performance and querying workload compared to autonomous and always-querying baselines and adjusts its querying behavior to account for higher workload in users with mobility limitations. We validate this through experiments in a simulated food dataset and a user study with 19 participants, including one with severe mobility limitations. Please check out our project website at: http://emprise.cs.cornell.edu/hilbiteacquisition/