Conversational AI: The Science Behind the Alexa Prize

TL;DR

The paper discusses the Alexa Prize, a university competition aimed at advancing conversational AI by developing socialbots capable of engaging humans on various topics for extended periods, leveraging real user data and innovative techniques.

Contribution

It presents the collaborative efforts and technological advancements made by university teams and the Alexa Prize team to improve conversational AI systems through a large-scale, real-world competition.

Findings

Development of socialbots that converse coherently for 20 minutes

Integration of state-of-the-art NLP techniques with novel strategies

Enhanced understanding of user engagement and system scalability

Abstract

Conversational agents are exploding in popularity. However, much work remains in the area of social conversation as well as free-form conversation over a broad range of domains and topics. To advance the state of the art in conversational AI, Amazon launched the Alexa Prize, a 2.5-million-dollar university competition where sixteen selected university teams were challenged to build conversational agents, known as socialbots, to converse coherently and engagingly with humans on popular topics such as Sports, Politics, Entertainment, Fashion and Technology for 20 minutes. The Alexa Prize offers the academic community a unique opportunity to perform research with a live system used by millions of users. The competition provided university teams with real user conversational data at scale, along with the user-provided ratings and feedback augmented with annotations by the Alexa team. This enabled teams to effectively iterate and make improvements throughout the competition while being evaluated in real-time through live user interactions. To build their socialbots, university teams combined state-of-the-art techniques with novel strategies in the areas of Natural Language Understanding, Context Modeling, Dialog Management, Response Generation, and Knowledge Acquisition. To support the efforts of participating teams, the Alexa Prize team made significant scientific and engineering investments to build and improve Conversational Speech Recognition, Topic Tracking, Dialog Evaluation, Voice User Experience, and tools for traffic management and scalability. This paper outlines the advances created by the university teams as well as the Alexa Prize team to achieve the common goal of solving the problem of Conversational AI.