Learning with Weak Supervision for Email Intent Detection

TL;DR

This paper introduces a deep learning approach for email intent detection that combines limited labeled data with noisy user action logs as weak supervision, improving performance in real-world scenarios with scarce annotations.

Contribution

It proposes a novel end-to-end neural network model that effectively integrates weak supervision from user actions with limited annotated data for email intent detection.

Findings

Weak supervision from user actions enhances intent detection accuracy.

The model outperforms baseline methods on multiple email intent tasks.

Self-paced learning improves the robustness of the model.

Abstract

Email remains one of the most frequently used means of online communication. People spend a significant amount of time every day on emails to exchange information, manage tasks and schedule events. Previous work has studied different ways for improving email productivity by prioritizing emails, suggesting automatic replies or identifying intents to recommend appropriate actions. The problem has been mostly posed as a supervised learning problem where models of different complexities were proposed to classify an email message into a predefined taxonomy of intents or classes. The need for labeled data has always been one of the largest bottlenecks in training supervised models. This is especially the case for many real-world tasks, such as email intent classification, where large scale annotated examples are either hard to acquire or unavailable due to privacy or data access constraints. Email users often take actions in response to intents expressed in an email (e.g., setting up a meeting in response to an email with a scheduling request). Such actions can be inferred from user interaction logs. In this paper, we propose to leverage user actions as a source of weak supervision, in addition to a limited set of annotated examples, to detect intents in emails. We develop an end-to-end robust deep neural network model for email intent identification that leverages both clean annotated data and noisy weak supervision along with a self-paced learning mechanism. Extensive experiments on three different intent detection tasks show that our approach can effectively leverage the weakly supervised data to improve intent detection in emails.