Introducing "Forecast Utterance" for Conversational Data Science

TL;DR

This paper introduces 'Forecast Utterance', a new way for conversational agents to understand user forecasting goals through zero-shot slot-filling methods, advancing natural language interfaces for data science tasks.

Contribution

It pioneers the concept of Forecast Utterance and demonstrates effective zero-shot methods for interpreting user goals in forecasting tasks.

Findings

Entity Extraction method effectively interprets forecast goals.

Question-Answering approach also shows high accuracy.

Both methods validate the feasibility of natural language forecasting interfaces.

Abstract

Envision an intelligent agent capable of assisting users in conducting forecasting tasks through intuitive, natural conversations, without requiring in-depth knowledge of the underlying machine learning (ML) processes. A significant challenge for the agent in this endeavor is to accurately comprehend the user's prediction goals and, consequently, formulate precise ML tasks. In this paper, we take a pioneering step towards this ambitious goal by introducing a new concept called Forecast Utterance and then focus on the automatic and accurate interpretation of users' prediction goals from these utterances. Specifically, we frame the task as a slot-filling problem, where each slot corresponds to a specific aspect of the goal prediction task. We then employ two zero-shot methods for solving the slot-filling task, namely: 1) Entity Extraction (EE), and 2) Question-Answering (QA) techniques. Our experiments, conducted with three meticulously crafted data sets, validate the viability of our ambitious goal and demonstrate the effectiveness of both EE and QA techniques in interpreting Forecast Utterances.