Using Information Theory to Measure Psychophysical Performance

TL;DR

This paper introduces a new information-theoretic measure called Shannon competence to quantify psychophysical performance by integrating reaction times and binary responses, revealing their synergistic contribution and advantages over traditional methods.

Contribution

It defines Shannon competence using mutual information, incorporating reaction times and binary responses with a covariance-aware diffusion model, improving measurement efficiency and robustness.

Findings

Reaction times provide independent information from binary responses.

Reaction times and binary responses together offer more information than separately.

Using combined data reduces the number of stimulus presentations needed.

Abstract

Most psychophysical experiments discard half the data collected. Specifically, experiments discard reaction time data, and use binary responses (e.g. yes/no) to measure performance. Here, Shannon's information theory is used to define Shannon competence $s'$, which depends on the mutual information between stimulus strength (e.g. luminance) and a combination of reaction times and binary responses. Mutual information is the entropy of the joint distribution of responses minus the residual entropy after a model has been fitted to these responses. Here, this model is instantiated as a proportional rate diffusion model, with the additional innovation that the full covariance structure of responses is taken into account. Results suggest information associated with reaction times is independent of (i.e. additional to) information associated with binary responses, and that reaction time and binary responses together provide substantially more than the sum of their individual contributions (i.e. they act synergistically). Consequently, the additional information supplied by reaction times suggests that using combined reaction time and binary responses requires fewer stimulus presentations, without loss of precision in psychophysical parameters. Finally, because s' takes account of both reaction time and binary responses, (and in contrast to d') $s'$ is immune to speed-accuracy trade-offs, which vary between observers and experimental designs.