Face valuing: Training user interfaces with facial expressions and reinforcement learning

TL;DR

This paper introduces face valuing, a method where a machine learns to interpret facial expressions to infer user preferences, reducing the need for explicit feedback in human-machine interaction.

Contribution

The work demonstrates that an agent can learn to perceive a value of its behavior from facial expressions, enabling quicker adaptation with less explicit feedback.

Findings

Agents can adapt to user preferences with minimal explicit feedback.

Facial features can be mapped to expected rewards in a task.

Face valuing complements existing interactive machine learning methods.

Abstract

An important application of interactive machine learning is extending or amplifying the cognitive and physical capabilities of a human. To accomplish this, machines need to learn about their human users' intentions and adapt to their preferences. In most current research, a user has conveyed preferences to a machine using explicit corrective or instructive feedback; explicit feedback imposes a cognitive load on the user and is expensive in terms of human effort. The primary objective of the current work is to demonstrate that a learning agent can reduce the amount of explicit feedback required for adapting to the user's preferences pertaining to a task by learning to perceive a value of its behavior from the human user, particularly from the user's facial expressions---we call this face valuing. We empirically evaluate face valuing on a grip selection task. Our preliminary results suggest that an agent can quickly adapt to a user's changing preferences with minimal explicit feedback by learning a value function that maps facial features extracted from a camera image to expected future reward. We believe that an agent learning to perceive a value from the body language of its human user is complementary to existing interactive machine learning approaches and will help in creating successful human-machine interactive applications.