Best Practices for Data-Efficient Modeling in NLG:How to Train Production-Ready Neural Models with Less Data

TL;DR

This paper presents practical techniques and best practices for training small, data-efficient neural language generation models suitable for production in conversational systems, addressing challenges like high data requirements and latency.

Contribution

It introduces a family of sampling and modeling techniques that enable deployment of small, efficient neural NLG models with limited data, along with a comprehensive set of best practices.

Findings

Domain complexity influences the choice of data-efficient approach.

Small models (2MB) can achieve production quality with limited data.

The techniques enable reliable deployment of neural NLG in production environments.

Abstract

Natural language generation (NLG) is a critical component in conversational systems, owing to its role of formulating a correct and natural text response. Traditionally, NLG components have been deployed using template-based solutions. Although neural network solutions recently developed in the research community have been shown to provide several benefits, deployment of such model-based solutions has been challenging due to high latency, correctness issues, and high data needs. In this paper, we present approaches that have helped us deploy data-efficient neural solutions for NLG in conversational systems to production. We describe a family of sampling and modeling techniques to attain production quality with light-weight neural network models using only a fraction of the data that would be necessary otherwise, and show a thorough comparison between each. Our results show that domain complexity dictates the appropriate approach to achieve high data efficiency. Finally, we distill the lessons from our experimental findings into a list of best practices for production-level NLG model development, and present them in a brief runbook. Importantly, the end products of all of the techniques are small sequence-to-sequence models (2Mb) that we can reliably deploy in production.