NEST: Neural Estimation by Sequential Testing

TL;DR

This paper introduces NEST, a neural network-based adaptive procedure for efficiently estimating complex multi-dimensional psychometric functions, outperforming existing methods without kernel selection.

Contribution

It proposes a novel neural estimation approach with a new acquisition function, eliminating the need for kernel tuning in non-parametric psychometric estimation.

Findings

Outperforms state-of-the-art methods in simulations and experiments

Reduces experiment duration significantly

Does not require kernel function selection

Abstract

Adaptive psychophysical procedures aim to increase the efficiency and reliability of measurements. With increasing stimulus and experiment complexity in the last decade, estimating multi-dimensional psychometric functions has become a challenging task for adaptive procedures. If the experimenter has limited information about the underlying psychometric function, it is not possible to use parametric techniques developed for the multi-dimensional stimulus space. Although there are non-parametric approaches that use Gaussian process methods and specific hand-crafted acquisition functions, their performance is sensitive to proper selection of the kernel function, which is not always straightforward. In this work, we use a neural network as the psychometric function estimator and introduce a novel acquisition function for stimulus selection. We thoroughly benchmark our technique both using simulations and by conducting psychovisual experiments under realistic conditions. We show that our method outperforms the state of the art without the need to select a kernel function and significantly reduces the experiment duration.