Zero-shot counting with a dual-stream neural network model

TL;DR

This paper introduces a dual-stream neural network model that mimics human-like zero-shot counting abilities by integrating dorsal and ventral pathways, forming spatial and numerical representations similar to primate brain activity.

Contribution

It presents a novel dual-stream architecture that generalizes counting to new items and models neural responses akin to macaque parietal cortex.

Findings

Model successfully generalizes counting to novel objects.

Reproduces neural response patterns observed in primate studies.

Predicts human gaze behavior during counting tasks.

Abstract

Deep neural networks have provided a computational framework for understanding object recognition, grounded in the neurophysiology of the primate ventral stream, but fail to account for how we process relational aspects of a scene. For example, deep neural networks fail at problems that involve enumerating the number of elements in an array, a problem that in humans relies on parietal cortex. Here, we build a 'dual-stream' neural network model which, equipped with both dorsal and ventral streams, can generalise its counting ability to wholly novel items ('zero-shot' counting). In doing so, it forms spatial response fields and lognormal number codes that resemble those observed in macaque posterior parietal cortex. We use the dual-stream network to make successful predictions about behavioural studies of the human gaze during similar counting tasks.