RTify: Aligning Deep Neural Networks with Human Behavioral Decisions

TL;DR

This paper introduces RTify, a framework that aligns neural network decision dynamics with human reaction times, improving models of human visual decision-making by integrating temporal behavioral data.

Contribution

The paper presents a novel method to constrain RNNs with human RTs, enabling better modeling of human perceptual decision dynamics and integrating these with visual processing models.

Findings

The model accurately predicts human reaction times across experiments.

The approach enables optimization of speed-accuracy tradeoffs in decision models.

Integration with CNNs improves natural image decision modeling.

Abstract

Current neural network models of primate vision focus on replicating overall levels of behavioral accuracy, often neglecting perceptual decisions' rich, dynamic nature. Here, we introduce a novel computational framework to model the dynamics of human behavioral choices by learning to align the temporal dynamics of a recurrent neural network (RNN) to human reaction times (RTs). We describe an approximation that allows us to constrain the number of time steps an RNN takes to solve a task with human RTs. The approach is extensively evaluated against various psychophysics experiments. We also show that the approximation can be used to optimize an "ideal-observer" RNN model to achieve an optimal tradeoff between speed and accuracy without human data. The resulting model is found to account well for human RT data. Finally, we use the approximation to train a deep learning implementation of the popular Wong-Wang decision-making model. The model is integrated with a convolutional neural network (CNN) model of visual processing and evaluated using both artificial and natural image stimuli. Overall, we present a novel framework that helps align current vision models with human behavior, bringing us closer to an integrated model of human vision.