Improving Formality Style Transfer with Context-Aware Rule Injection
Zonghai Yao, Hong Yu

TL;DR
This paper introduces CARI, a novel context-aware rule injection method for formality style transfer that enhances BERT-based models, significantly improving performance on style transfer benchmarks and sentiment analysis tasks.
Contribution
CARI is the first method to inject multiple context-aware rules into BERT models for formality style transfer, improving both style transfer quality and downstream sentiment analysis.
Findings
Achieved new highest performance on FST benchmark dataset.
Significantly improved sentiment analysis on tweets.
Demonstrated effectiveness of rule injection in style transfer.
Abstract
Models pre-trained on large-scale regular text corpora often do not work well for user-generated data where the language styles differ significantly from the mainstream text. Here we present Context-Aware Rule Injection (CARI), an innovative method for formality style transfer (FST). CARI injects multiple rules into an end-to-end BERT-based encoder and decoder model. It learns to select optimal rules based on context. The intrinsic evaluation showed that CARI achieved the new highest performance on the FST benchmark dataset. Our extrinsic evaluation showed that CARI can greatly improve the regular pre-trained models' performance on several tweet sentiment analysis tasks.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Sentiment Analysis and Opinion Mining
