Testing for a general changepoint in psychometric studies: changes detection and sample size planning

TL;DR

This paper presents a new, simple change detection method for psychometric studies using the pseudo Score statistic, enabling easier power analysis and sample size planning, with demonstrated superior performance over existing tests.

Contribution

The paper introduces a novel change detection approach based on the pseudo Score statistic, simplifying computation and facilitating power analysis in psychometric research.

Findings

Outperforms existing change point tests in simulations

Provides a practical tool available on CRAN and Shiny App

Effective in both normally distributed and binary data

Abstract

This paper introduces a new method for change detection in psychometric studies based on the recently introduced pseudo Score statistic, for which the sampling distribution under the alternative hypothesis has been determined. Our approach has the advantage of simplicity in its computation, eliminating the need for resampling or simulations to obtain critical values. Additionally, it comes with a known null/alternative distribution, facilitating easy calculations for power levels and sample size planning. The paper indeed also discusses the topic of power analysis in segmented regression, namely the estimation of sample size or power level when the study data being collected focuses on a covariate expected to affect the mean response via a piecewise relationship with an unknown breakpoint. We run simulation results showing that our method outperforms other Tests for a Change Point (TFCP) with both normally distributed and binary data and carry out a real SAT Critical reading data analysis. The proposed test contributes to the framework of psychometric research, and it is available on the Comprehensive R Archive Network (CRAN) and in a more user-friendly Shiny App, both illustrated at the end of the paper.