CommentScope: A Comment-Embedded Assisted Reading System for a Long Text

TL;DR

CommentScope is an innovative system that embeds comments within long texts, enhancing comprehension and comment discovery through classification and visual integration, supported by a fine-tuned LLM.

Contribution

It introduces a novel comment embedding approach with a classification pipeline and visual presentation module, improving reading experience and comment relevance understanding.

Findings

Semantic classification accuracy=0.89

Position exact match=0.82

Improved comment discovery and reading fluency in user study

Abstract

Long texts are ubiquitous on social platforms, yet readers often face information overload and struggle to locate key content. Comments provide valuable external perspectives for understanding, questioning, and complementing the text, but their potential is hindered by disorganized and unstructured presentation. Few studies have explored embedding comments directly into reading. As an exploratory step, we propose CommentScope, a system with two core modules: a classification pipeline powered by a fine-tuned Large Language Model (LLM) that categorizes comments into five pragmatic types and aligns them with relevant sentences, and a presentation module that integrates comments inline or as side notes, supported by visual cues like colors, charts, and highlights. Technical evaluation demonstrates that the fine-tuned model effectively captures implicit pragmatic functions and context, achieving solid performance in semantic classification (accuracy=0.89) and position exact match (EM=0.82). A user study (N=12) further demonstrates that the sentence-end embedding improved comment discovery accuracy and reading fluency, while reducing mental demand and perceived effort compared to traditional baselines.