Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks

TL;DR

This survey reviews recent machine learning frameworks that leverage human guidance beyond traditional reward functions and demonstrations to improve sequential decision-making in artificial agents.

Contribution

It provides a high-level overview of five recent frameworks that utilize alternative human guidance methods in sequential decision-making tasks.

Findings

Identifies five recent frameworks using human guidance

Discusses motivations and assumptions of each framework

Highlights future research directions in human-guided learning

Abstract

A longstanding goal of artificial intelligence is to create artificial agents capable of learning to perform tasks that require sequential decision making. Importantly, while it is the artificial agent that learns and acts, it is still up to humans to specify the particular task to be performed. Classical task-specification approaches typically involve humans providing stationary reward functions or explicit demonstrations of the desired tasks. However, there has recently been a great deal of research energy invested in exploring alternative ways in which humans may guide learning agents that may, e.g., be more suitable for certain tasks or require less human effort. This survey provides a high-level overview of five recent machine learning frameworks that primarily rely on human guidance apart from pre-specified reward functions or conventional, step-by-step action demonstrations. We review the motivation, assumptions, and implementation of each framework, and we discuss possible future research directions.