# Performance of normative and approximate evidence accumulation on the dynamic clicks task

## TL;DR

This paper investigates how normative and approximate evidence accumulation models perform in a dynamic clicks task, revealing conditions for optimality, model distinguishability, and implications for experimental design and data interpretation.

## Contribution

It introduces a detailed analysis of ideal and near-ideal observers in a dynamic decision task, highlighting how model tuning and fitting methods affect performance assessment.

## Key findings

- Optimal performance regions depend on specific task parameters.
- Approximate models require fine-tuning to achieve near-optimal results.
- Using 0/1-loss for model fitting introduces bias, especially with sensory noise.

## Abstract

The aim of a number of psychophysics tasks is to uncover how mammals make decisions in a world that is in flux. Here we examine the characteristics of ideal and near-ideal observers in a task of this type. We ask when and how performance depends on task parameters and design, and, in turn, what observer performance tells us about their decision-making process. In the dynamic clicks task subjects hear two streams (left and right) of Poisson clicks with different rates. Subjects are rewarded when they correctly identify the side with the higher rate, as this side switches unpredictably. We show that a reduced set of task parameters defines regions in parameter space in which optimal, but not near-optimal observers, maintain constant response accuracy. We also show that for a range of task parameters an approximate normative model must be finely tuned to reach near-optimal performance, illustrating a potential way to distinguish between normative models and their approximations. In addition, we show that using the negative log-likelihood and the 0/1-loss functions to fit these types of models is not equivalent: the 0/1-loss leads to a bias in parameter recovery that increases with sensory noise. These findings suggest ways to tease apart models that are hard to distinguish when tuned exactly, and point to general pitfalls in experimental design, model fitting, and interpretation of the resulting data.