Fitting, Evaluating, and Comparing Cognitive Architecture Models Using Likelihood: A Primer With Examples in ACT-R

TL;DR

This paper introduces likelihood-based methods for fitting, evaluating, and comparing cognitive architecture models, specifically using ACT-R, emphasizing their importance for model validation and providing practical Python tools.

Contribution

It provides a comprehensive primer on applying maximum likelihood techniques to cognitive models in ACT-R, including implementation guidance and technical references.

Findings

Likelihood methods improve model validation in cognitive architectures

Practical Python notebook facilitates implementation of likelihood-based techniques

Guidelines for applying likelihood measures to models of varying complexity

Abstract

Cognitive architectures are influential, integrated computational frameworks for modeling cognitive processes. Due to a variety of factors, however, researchers using cognitive architectures to explain and predict human performance rarely employ model validation, comparison, and selection techniques based on likelihood. This paper provides a primer on how to implement maximum likelihood techniques and its derivatives to fit and compare models at the individual and group level, using models implemented in the ACT-R cognitive architecture as examples. The paper covers the most common ways in which likelihood measures can be applied, under different scenarios, for models of different complexity, and provides further technical references for the interested reader. An accompanying notebook in Python provides the code to implement all of the suggestions.