DeepProbe: Information Directed Sequence Understanding and Chatbot Design via Recurrent Neural Networks

TL;DR

DeepProbe introduces an attention-based seq2seq framework for enhanced query understanding and chatbot interactions, enabling query rewriting, relevance scoring, and active user engagement through uncertainty-driven decision making.

Contribution

The paper presents a novel, unified seq2seq-based framework for information extraction and interactive chatbot design, with significant improvements over existing methods.

Findings

DeepProbe outperforms baselines in BLEU and AUC metrics.

Human evaluations favor DeepProbe's applications over state-of-the-art systems.

Active questioning improves user intention identification efficiency.

Abstract

Information extraction and user intention identification are central topics in modern query understanding and recommendation systems. In this paper, we propose DeepProbe, a generic information-directed interaction framework which is built around an attention-based sequence to sequence (seq2seq) recurrent neural network. DeepProbe can rephrase, evaluate, and even actively ask questions, leveraging the generative ability and likelihood estimation made possible by seq2seq models. DeepProbe makes decisions based on a derived uncertainty (entropy) measure conditioned on user inputs, possibly with multiple rounds of interactions. Three applications, namely a rewritter, a relevance scorer and a chatbot for ad recommendation, were built around DeepProbe, with the first two serving as precursory building blocks for the third. We first use the seq2seq model in DeepProbe to rewrite a user query into one of standard query form, which is submitted to an ordinary recommendation system. Secondly, we evaluate DeepProbe's seq2seq model-based relevance scoring. Finally, we build a chatbot prototype capable of making active user interactions, which can ask questions that maximize information gain, allowing for a more efficient user intention idenfication process. We evaluate first two applications by 1) comparing with baselines by BLEU and AUC, and 2) human judge evaluation. Both demonstrate significant improvements compared with current state-of-the-art systems, proving their values as useful tools on their own, and at the same time laying a good foundation for the ongoing chatbot application.