Inconsistent illusory motion in predictive coding deep neural networks

TL;DR

This study investigates whether predictive coding deep neural networks can reliably reproduce human-like illusory motion perceptions, revealing inconsistencies and limitations in current models despite some similarities.

Contribution

The paper replicates previous findings that PredNet can predict illusory motion and critically examines its behavior, uncovering variability and discrepancies with human perception.

Findings

PredNet predicts illusory motion in all stimulus components.

No simple response delays in internal units, unlike physiological data.

Large variation in illusion reproduction across different trained networks.

Abstract

Why do we perceive illusory motion in some static images? Several accounts have been proposed based on eye movements, response latencies to different image elements, or interactions between image patterns and motion energy detectors. Recently, PredNet, a recurrent deep neural network (DNN) based on predictive coding principles, was reported to reproduce the "Rotating Snakes" illusion, suggesting a role for predictive coding in illusory motion. We replicate this finding and then use a series of "in silico psychophysics" experiments to examine whether PredNet behaves consistently with human observers for simplified variants of the illusory stimuli. We also measure response latencies to individual elements of the Rotating Snakes pattern by probing internal units in the network. A pretrained PredNet model predicted illusory motion for all subcomponents of the Rotating Snakes stimulus, consistent with human observers. However, we found no simple response delays in internal units, as found in physiological data. The PredNet model's detection of motion in gradients was based on contrast, not luminance, as it is in human perception. Finally, we tested the robustness of the illusion on 10 identical PredNets trained on the same video data; we found a large variation in the ability of the network to reproduce the illusion and predict motion for simplified variants of the illusion. Also, unlike human observers, none of the networks predicted illusory motion for greyscale variants of the pattern. Even when a DNN successfully reproduces some idiosyncrasy of human vision, a more detailed investigation can reveal inconsistencies between humans and the network and between different instances of the same network. The inconsistency of the Rotating Snakes illusion in PredNets trained from different initializations suggests that predictive coding does not reliably lead to human-like illusory motion.