Deep Epitome for Unravelling Generalized Hamming Network: A Fuzzy Logic Interpretation of Deep Learning

TL;DR

This paper provides a theoretical analysis of Generalized Hamming Networks, revealing their equivalence to wide convolution layers, and introduces deep epitomes for visualization and feature extraction without input data dependence.

Contribution

It offers a rigorous theoretical understanding of GHNs and introduces deep epitomes for visualization and feature extraction independent of input data.

Findings

Stacked convolution layers in GHNs are equivalent to a single wide convolution layer.

Deep epitomes enable visualization of internal representations without input data.

Features can be extracted directly using deep epitomes without regularized optimization.

Abstract

This paper gives a rigorous analysis of trained Generalized Hamming Networks(GHN) proposed by Fan (2017) and discloses an interesting finding about GHNs, i.e., stacked convolution layers in a GHN is equivalent to a single yet wide convolution layer. The revealed equivalence, on the theoretical side, can be regarded as a constructive manifestation of the universal approximation theorem Cybenko(1989); Hornik (1991). In practice, it has profound and multi-fold implications. For network visualization, the constructed deep epitomes at each layer provide a visualization of network internal representation that does not rely on the input data. Moreover, deep epitomes allows the direct extraction of features in just one step, without resorting to regularized optimizations used in existing visualization tools.