Modeling Text Complexity using a Multi-Scale Probit

TL;DR

This paper introduces a multi-scale probit model for analyzing text complexity across different annotation schemes, effectively combining diverse corpora to improve predictive accuracy in readability assessment.

Contribution

The paper presents a novel multi-scale probit model with a Gibbs sampler for text complexity analysis, enabling integration of heterogeneous annotation data.

Findings

Effective combination of multiple corpora with different annotation schemes.

Promising predictive performance on simulated and real readability data.

Addresses the p>n problem in text complexity modeling.

Abstract

We present a novel model for text complexity analysis which can be fitted to ordered categorical data measured on multiple scales, e.g. a corpus with binary responses mixed with a corpus with more than two ordered outcomes. The multiple scales are assumed to be driven by the same underlying latent variable describing the complexity of the text. We propose an easily implemented Gibbs sampler to sample from the posterior distribution by a direct extension of established data augmentation schemes. By being able to combine multiple corpora with different annotation schemes we can get around the common problem of having more text features than annotated documents, i.e. an example of the $p>n$ problem. The predictive performance of the model is evaluated using both simulated and real world readability data with very promising results.