Towards A Virtual Assistant That Can Be Taught New Tasks In Any Domain By Its End-Users

TL;DR

This paper introduces Helpa, a domain-independent virtual assistant that can be taught new tasks by end-users through a single command and demonstration, without programming, enabling quick and easy task learning.

Contribution

The paper presents Helpa, a novel instructible virtual assistant system that learns new tasks from natural language commands with minimal user effort and no domain-specific knowledge.

Findings

End-users can teach Helpa new tasks in less than a minute.

Helpa successfully learns tasks with only one example of command and demonstration.

Helpa operates without relying on pre-existing domain knowledge.

Abstract

The challenge stated in the title can be divided into two main problems. The first problem is to reliably mimic the way that users interact with user interfaces. The second problem is to build an instructible agent, i.e. one that can be taught to execute tasks expressed as previously unseen natural language commands. This paper proposes a solution to the second problem, a system we call Helpa. End-users can teach Helpa arbitrary new tasks whose level of complexity is similar to the tasks available from today's most popular virtual assistants. Teaching Helpa does not involve any programming. Instead, users teach Helpa by providing just one example of a command paired with a demonstration of how to execute that command. Helpa does not rely on any pre-existing domain-specific knowledge. It is therefore completely domain-independent. Our usability study showed that end-users can teach Helpa many new tasks in less than a minute each, often much less.