# Learning User Preferences via Reinforcement Learning with Spatial Interface Valuing

## TL;DR

This paper introduces Spatial Interface Valuing, a method where agents learn user preferences through hand gestures, reducing explicit feedback and improving learning speed in interactive machine learning tasks.

## Contribution

The study demonstrates that agents can effectively learn from spatial interface gestures, decreasing reliance on explicit feedback and enhancing learning efficiency in human-agent interactions.

## Key findings

- Spatial interface feedback accelerates learning.
- Agents can map gestures to expected rewards.
- Reduced explicit feedback improves interaction efficiency.

## Abstract

Interactive Machine Learning is concerned with creating systems that operate in environments alongside humans to achieve a task. A typical use is to extend or amplify the capabilities of a human in cognitive or physical ways, requiring the machine to adapt to the users' intentions and preferences. Often, this takes the form of a human operator providing some type of feedback to the user, which can be explicit feedback, implicit feedback, or a combination of both. Explicit feedback, such as through a mouse click, carries a high cognitive load. The focus of this study is to extend the current state of the art in interactive machine learning by demonstrating that agents can learn a human user's behavior and adapt to preferences with a reduced amount of explicit human feedback in a mixed feedback setting. The learning agent perceives a value of its own behavior from hand gestures given via a spatial interface. This feedback mechanism is termed Spatial Interface Valuing. This method is evaluated experimentally in a simulated environment for a grasping task using a robotic arm with variable grip settings. Preliminary results indicate that learning agents using spatial interface valuing can learn a value function mapping spatial gestures to expected future rewards much more quickly as compared to those same agents just receiving explicit feedback, demonstrating that an agent perceiving feedback from a human user via a spatial interface can serve as an effective complement to existing approaches.