Profiling Visual Dynamic Complexity Using a Bio-Robotic Approach

TL;DR

This paper introduces a novel bio-robotic method using a brain-inspired neural network to quantify visual dynamic complexity, enabling better understanding and prediction of scene difficulty for dynamic vision systems.

Contribution

It presents the first implementation of a brain-inspired motion detection model in an autonomous robot to profile and quantify visual dynamic complexity from spatial-temporal features.

Findings

The neural response correlates monotonically with scene complexity.

The approach is flexible across different visual scenes.

The metric predicts the boundary of collision detection system performance.

Abstract

Visual dynamic complexity is a ubiquitous, hidden attribute of the visual world that every dynamic vision system is faced with. However, it is implicit and intractable which has never been quantitatively described due to the difficulty in defending temporal features correlated to spatial image complexity. To fill this vacancy, we propose a novel bio-robotic approach to profile visual dynamic complexity which can be used as a new metric. Here we apply a state-of-the-art brain-inspired motion detection neural network model to explicitly profile such complexity associated with spatial-temporal frequency (SF-TF) of visual scene. This model is for the first time implemented in an autonomous micro-mobile robot which navigates freely in an arena with visual walls displaying moving sine-wave grating or cluttered natural scene. The neural dynamic response can make reasonable prediction on surrounding complexity since it can be mapped monotonically to varying SF-TF of visual scene. The experiments show this approach is flexible to different visual scenes for profiling the dynamic complexity. We also use this metric as a predictor to investigate the boundary of another collision detection visual system performing in changing environment with increasing dynamic complexity. This research demonstrates a new paradigm of using biologically plausible visual processing scheme to estimate dynamic complexity of visual scene from both spatial and temporal perspectives, which could be beneficial to predicting input complexity when evaluating dynamic vision systems.