A Dataset and Baselines for Multilingual Reply Suggestion

TL;DR

This paper introduces MRS, a multilingual reply suggestion dataset covering ten languages, along with baseline models, to advance research in multilingual email and chat reply generation and retrieval.

Contribution

It provides the first multilingual reply suggestion dataset and baseline models, enabling cross-lingual research beyond English.

Findings

Retrieval models excel in monolingual settings.

Generation models produce more natural replies.

Different strategies are needed for cross-lingual generalization.

Abstract

Reply suggestion models help users process emails and chats faster. Previous work only studies English reply suggestion. Instead, we present MRS, a multilingual reply suggestion dataset with ten languages. MRS can be used to compare two families of models: 1) retrieval models that select the reply from a fixed set and 2) generation models that produce the reply from scratch. Therefore, MRS complements existing cross-lingual generalization benchmarks that focus on classification and sequence labeling tasks. We build a generation model and a retrieval model as baselines for MRS. The two models have different strengths in the monolingual setting, and they require different strategies to generalize across languages. MRS is publicly available at https://github.com/zhangmozhi/mrs.