Incorporating Stylistic Lexical Preferences in Generative Language Models

TL;DR

This paper introduces a reinforcement learning method to incorporate an author's lexical style preferences into transformer-based language models, enabling more personalized and stylistically consistent text generation.

Contribution

It presents a novel approach to explicitly encode and induce target-author lexical preferences in language models using reinforcement learning.

Findings

Generated text aligns with target author's lexical style

The approach outperforms baseline models in style consistency

Both quantitative and qualitative evaluations support effectiveness

Abstract

While recent advances in language modeling have resulted in powerful generation models, their generation style remains implicitly dependent on the training data and can not emulate a specific target style. Leveraging the generative capabilities of a transformer-based language models, we present an approach to induce certain target-author attributes by incorporating continuous multi-dimensional lexical preferences of an author into generative language models. We introduce rewarding strategies in a reinforcement learning framework that encourages the use of words across multiple categorical dimensions, to varying extents. Our experiments demonstrate that the proposed approach can generate text that distinctively aligns with a given target author's lexical style. We conduct quantitative and qualitative comparisons with competitive and relevant baselines to illustrate the benefits of the proposed approach.