Conciseness: An Overlooked Language Task
Felix Stahlberg, Aashish Kumar, Chris Alberti, Shankar Kumar

TL;DR
This paper introduces the task of sentence conciseness, differentiates it from related tasks, and proposes a synthetic data generation method to improve model performance, highlighting its difficulty for large neural models and providing new datasets and baselines.
Contribution
The paper defines sentence conciseness as a distinct task, releases new annotated test sets, and proposes a synthetic data generation approach to enhance model training.
Findings
Large neural models struggle with zero-shot conciseness tasks.
Synthetic data improves model performance on conciseness.
Fine-tuning on artificial datasets yields strong baselines.
Abstract
We report on novel investigations into training models that make sentences concise. We define the task and show that it is different from related tasks such as summarization and simplification. For evaluation, we release two test sets, consisting of 2000 sentences each, that were annotated by two and five human annotators, respectively. We demonstrate that conciseness is a difficult task for which zero-shot setups with large neural language models often do not perform well. Given the limitations of these approaches, we propose a synthetic data generation method based on round-trip translations. Using this data to either train Transformers from scratch or fine-tune T5 models yields our strongest baselines that can be further improved by fine-tuning on an artificial conciseness dataset that we derived from multi-annotator machine translation test sets.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Text Readability and Simplification
MethodsMulti-Head Attention · Attention Is All You Need · Test · Linear Layer · Byte Pair Encoding · Residual Connection · Dropout · SentencePiece · Attention Dropout · Adafactor
