Understanding Deep Convolutional Networks through Gestalt Theory

TL;DR

This paper introduces a Gestalt theory-based sensitivity analysis to better understand how deep convolutional networks process visual information, revealing that they follow many Gestalt principles and can be made more explainable.

Contribution

A novel framework applying Gestalt principles to analyze and interpret the internal mechanisms of deep convolutional networks.

Findings

ConvNets follow Gestalt perceptual principles at multiple levels

The framework stimulates specific feature maps for better understanding

Reveals important network attributes for explainability

Abstract

The superior performance of deep convolutional networks over high-dimensional problems have made them very popular for several applications. Despite their wide adoption, their underlying mechanisms still remain unclear with their improvement procedures still relying mainly on a trial and error process. We introduce a novel sensitivity analysis based on the Gestalt theory for giving insights into the classifier function and intermediate layers. Since Gestalt psychology stipulates that perception can be a product of complex interactions among several elements, we perform an ablation study based on this concept to discover which principles and image context significantly contribute in the network classification. Our results reveal that ConvNets follow most of the visual cortical perceptual mechanisms defined by the Gestalt principles at several levels. The proposed framework stimulates specific feature maps in classification problems and reveal important network attributes that can produce more explainable network models.