BBScore: A Brownian Bridge Based Metric for Assessing Text Coherence
Zhecheng Sheng, Tianhao Zhang, Chen Jiang, Dongyeop Kang

TL;DR
The paper introduces BBScore, a novel reference-free metric based on Brownian bridge theory, for assessing both local and global text coherence, demonstrating competitive performance and applicability in distinguishing human and AI-generated texts.
Contribution
It proposes a new coherence metric that captures overarching text coherence without relying on static embeddings or end-to-end training, enhancing evaluation of long texts.
Findings
Achieves performance comparable to state-of-the-art methods on artificial discrimination tasks.
Effectively differentiates between human-written and AI-generated texts in downstream applications.
Demonstrates potential for generalization across various large language model styles.
Abstract
Measuring the coherence of text is a vital aspect of evaluating the quality of written content. Recent advancements in neural coherence modeling have demonstrated their efficacy in capturing entity coreference and discourse relations, thereby enhancing coherence evaluation. However, many existing methods heavily depend on static embeddings or focus narrowly on nearby context, constraining their capacity to measure the overarching coherence of long texts. In this paper, we posit that coherent texts inherently manifest a sequential and cohesive interplay among sentences, effectively conveying the central theme, purpose, or standpoint. To explore this abstract relationship, we introduce the "BBScore," a novel reference-free metric grounded in Brownian bridge theory for assessing text coherence. Our findings showcase that when synergized with a simple additional classification component,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Text Readability and Simplification
MethodsFocus
