Putting Natural in Natural Language Processing

TL;DR

This paper advocates for a shift in NLP to prioritize spoken language processing, emphasizing the integration of speech and text to develop more natural, human-like communication systems.

Contribution

It highlights the need to unify speech and text processing in NLP, proposing a paradigm shift towards treating spoken language as primary.

Findings

Recent deep learning advances enable convergence of speech and NLP methods.

Unifying speech and NLP can improve data efficiency and human-likeness.

Prioritizing spoken language processing can enhance language science integration.

Abstract

Human language is firstly spoken and only secondarily written. Text, however, is a very convenient and efficient representation of language, and modern civilization has made it ubiquitous. Thus the field of NLP has overwhelmingly focused on processing written rather than spoken language. Work on spoken language, on the other hand, has been siloed off within the largely separate speech processing community which has been inordinately preoccupied with transcribing speech into text. Recent advances in deep learning have led to a fortuitous convergence in methods between speech processing and mainstream NLP. Arguably, the time is ripe for a unification of these two fields, and for starting to take spoken language seriously as the primary mode of human communication. Truly natural language processing could lead to better integration with the rest of language science and could lead to systems which are more data-efficient and more human-like, and which can communicate beyond the textual modality.