Generation, Evaluation, and Explanation of Novelists' Styles with Single-Token Prompts

TL;DR

This paper introduces a framework for generating and evaluating 19th-century novelists' styles using minimal prompts and AI-based methods, addressing challenges in stylometry without paired data and human-only evaluation.

Contribution

It presents a novel approach combining single-token prompts for style generation with transformer-based evaluation and explainability techniques, advancing stylometry research.

Findings

Generated text captures distinctive author styles

AI evaluation correlates well with human judgment

Explainability methods identify key stylistic features

Abstract

Recent advances in large language models have created new opportunities for stylometry, the study of writing styles and authorship. Two challenges, however, remain central: training generative models when no paired data exist, and evaluating stylistic text without relying only on human judgment. In this work, we present a framework for both generating and evaluating sentences in the style of 19th-century novelists. Large language models are fine-tuned with minimal, single-token prompts to produce text in the voices of authors such as Dickens, Austen, Twain, Alcott, and Melville. To assess these generative models, we employ a transformer-based detector trained on authentic sentences, using it both as a classifier and as a tool for stylistic explanation. We complement this with syntactic comparisons and explainable AI methods, including attention-based and gradient-based analyses, to identify the linguistic cues that drive stylistic imitation. Our findings show that the generated text reflects the authors' distinctive patterns and that AI-based evaluation offers a reliable alternative to human assessment. All artifacts of this work are published online.