Challenge of Spatial Cognition for Deep Learning

TL;DR

This paper investigates the limitations of deep convolutional neural networks in learning spatial concepts and proposes task-specific kernels to improve their generalization in spatial cognition tasks.

Contribution

It identifies the failure of standard DCNNs in spatial reasoning due to superficial learning and introduces task-specific convolutional kernels to enhance their spatial understanding.

Findings

Standard DCNNs struggle with spatial concepts across visual variations.

Incorporating task-specific kernels improves generalization to new visual inputs.

Manual priors are necessary for effective spatial cognition in deep learning models.

Abstract

Given the success of the deep convolutional neural networks (DCNNs) in applications of visual recognition and classification, it would be tantalizing to test if DCNNs can also learn spatial concepts, such as straightness, convexity, left/right, front/back, relative size, aspect ratio, polygons, etc., from varied visual examples of these concepts that are simple and yet vital for spatial reasoning. Much to our dismay, extensive experiments of the type of cognitive psychology demonstrate that the data-driven deep learning (DL) cannot see through superficial variations in visual representations and grasp the spatial concept in abstraction. The root cause of failure turns out to be the learning methodology, not the computational model of the neural network itself. By incorporating task-specific convolutional kernels, we are able to construct DCNNs for spatial cognition tasks that can generalize to input images not drawn from the same distribution of the training set. This work raises a precaution that without manually-incorporated priors or features DCCNs may fail spatial cognitive tasks at rudimentary level.