A note on the modeling of the effects of experimental time in psycholinguistic experiments

TL;DR

This paper clarifies the proper use of GAMMs in psycholinguistic experiments, demonstrating that with correct implementation and recent software improvements, GAMMs are reliable for modeling time effects, challenging prior criticisms.

Contribution

The authors show that using by-smooths in GAMMs avoids issues identified with factor smooths and highlight software bug fixes, supporting GAMMs as effective tools for analyzing time-dependent effects.

Findings

GAMMs with by-smooths produce correct estimates in simulations.

Software bug fix in mgcv improves GAMM reliability.

GAMMs outperform LMM in variability of estimates.

Abstract

Thul et al. (2020) called attention to problems that arise when chronometric experiments implementing specific factorial designs are analysed with the generalized additive mixed model (GAMM), using factor smooths to capture trial-to-trial dependencies. From a series of simulations incorporating such dependencies, they conclude that GAMMs are inappropriate for between-subject designs. They argue that in addition GAMMs come with too many modeling possibilities, and advise using the linear mixed model (LMM) instead. We address the questions raised by Thul et al. (2020), who clearly demonstrated that problems can indeed arise when using factor smooths in combination with factorial designs. We show that the problem does not arise when using by-smooths. Furthermore, we have traced a bug in the implementation of factor smooths in the mgcv package, which will have been removed from version 1.8-36 onwards. To illustrate that GAMMs now produce correct estimates, we report simulation studies implementing different by-subject longitudinal effects. The maximal LMM emerges as slightly conservative compared to GAMMs, and GAMMs provide estimated coefficients that can be less variable across simulation runs. We also discuss two datasets where time-varying effects interact with numerical predictors in a theoretically informative way. Furthermore, we argue that the wide range of tools that GAMMs make available to researcher across all domains of scientific inquiry do not come with uncontrolled researcher degrees of freedom once confronted with a specific psycholinguistic datasets. We also introduce a distinction between replicable and non-replicable non-linear effects. We conclude that GAMMs are an excellent and reliable tool for understanding experimental data, including chronometric data with time-varying effects.