Leveraging Systematic Knowledge of 2D Transformations

TL;DR

This paper introduces a new training methodology and architecture that enable deep neural networks to better understand and utilize 2D transformations, improving out-of-distribution performance in image classification.

Contribution

It presents a systematic knowledge acquisition method for 2D transformations and a novel CED architecture that emulates human perception to enhance robustness.

Findings

Networks trained with the new methodology show systematicity in parameter estimation.

The CED architecture significantly improves classification accuracy under covariate shift.

The approach demonstrates better out-of-distribution generalization compared to baseline models.

Abstract

The existing deep learning models suffer from out-of-distribution (o.o.d.) performance drop in computer vision tasks. In comparison, humans have a remarkable ability to interpret images, even if the scenes in the images are rare, thanks to the systematicity of acquired knowledge. This work focuses on 1) the acquisition of systematic knowledge of 2D transformations, and 2) architectural components that can leverage the learned knowledge in image classification tasks in an o.o.d. setting. With a new training methodology based on synthetic datasets that are constructed under the causal framework, the deep neural networks acquire knowledge from semantically different domains (e.g. even from noise), and exhibit certain level of systematicity in parameter estimation experiments. Based on this, a novel architecture is devised consisting of a classifier, an estimator and an identifier (abbreviated as "CED"). By emulating the "hypothesis-verification" process in human visual perception, CED improves the classification accuracy significantly on test sets under covariate shift.