Team "better_call_claude": Style Change Detection using a Sequential Sentence Pair Classifier

TL;DR

This paper introduces a Sequential Sentence Pair Classifier that effectively detects style changes at the sentence level in documents, leveraging pre-trained language models and contextual modeling to outperform baselines in a challenging benchmark.

Contribution

The paper presents a lightweight, effective approach using PLMs and BiLSTM for fine-grained style change detection, advancing the state-of-the-art in the PAN 2025 shared task.

Findings

Achieved macro-F1 scores of 0.923, 0.828, and 0.724 on EASY, MEDIUM, and HARD datasets.

Outperformed official baselines and zero-shot Claude-3.7 performance.

Demonstrated effectiveness in handling stylistically shallow, short sentences.

Abstract

Style change detection - identifying the points in a document where writing style shifts - remains one of the most important and challenging problems in computational authorship analysis. At PAN 2025, the shared task challenges participants to detect style switches at the most fine-grained level: individual sentences. The task spans three datasets, each designed with controlled and increasing thematic variety within documents. We propose to address this problem by modeling the content of each problem instance - that is, a series of sentences - as a whole, using a Sequential Sentence Pair Classifier (SSPC). The architecture leverages a pre-trained language model (PLM) to obtain representations of individual sentences, which are then fed into a bidirectional LSTM (BiLSTM) to contextualize them within the document. The BiLSTM-produced vectors of adjacent sentences are concatenated and passed to a multi-layer perceptron for prediction per adjacency. Building on the work of previous PAN participants classical text segmentation, the approach is relatively conservative and lightweight. Nevertheless, it proves effective in leveraging contextual information and addressing what is arguably the most challenging aspect of this year's shared task: the notorious problem of "stylistically shallow", short sentences that are prevalent in the proposed benchmark data. Evaluated on the official PAN-2025 test datasets, the model achieves strong macro-F1 scores of 0.923, 0.828, and 0.724 on the EASY, MEDIUM, and HARD data, respectively, outperforming not only the official random baselines but also a much more challenging one: claude-3.7-sonnet's zero-shot performance.