Inference Time Style Control for Summarization
Shuyang Cao, Lu Wang

TL;DR
This paper introduces two novel decoding methods for style control in summarization that do not require style-specific training data, enabling flexible and interpretable style adjustments during generation.
Contribution
The paper proposes decoder state adjustment and word prediction constraints as new techniques for style control in pre-trained summarization models without additional style-specific training.
Findings
Models produce simpler, informative summaries with style control.
Generated headlines exhibit distinguishable ideological leanings.
Methods are effective across different summarization styles.
Abstract
How to generate summaries of different styles without requiring corpora in the target styles, or training separate models? We present two novel methods that can be deployed during summary decoding on any pre-trained Transformer-based summarization model. (1) Decoder state adjustment instantly modifies decoder final states with externally trained style scorers, to iteratively refine the output against a target style. (2) Word unit prediction constrains the word usage to impose strong lexical control during generation. In experiments of summarizing with simplicity control, automatic evaluation and human judges both find our models producing outputs in simpler languages while still informative. We also generate news headlines with various ideological leanings, which can be distinguished by humans with a reasonable probability.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Advanced Text Analysis Techniques
