Tensor Normalization and Full Distribution Training

TL;DR

This paper introduces pixel-wise tensor normalization combined with multi-label reformulation to improve accuracy and robustness in deep neural networks, demonstrating significant empirical gains over traditional methods.

Contribution

It proposes a novel pixel-wise tensor normalization technique and a multi-label reformulation that enhance both accuracy and robustness of neural networks.

Findings

Tensor normalization improves network accuracy.

Multi-label reformulation increases robustness.

Combined approach outperforms traditional normalization methods.

Abstract

In this work, we introduce pixel wise tensor normalization, which is inserted after rectifier linear units and, together with batch normalization, provides a significant improvement in the accuracy of modern deep neural networks. In addition, this work deals with the robustness of networks. We show that the factorized superposition of images from the training set and the reformulation of the multi class problem into a multi-label problem yields significantly more robust networks. The reformulation and the adjustment of the multi class log loss also improves the results compared to the overlay with only one class as label. https://atreus.informatik.uni-tuebingen.de/seafile/d/8e2ab8c3fdd444e1a135/?p=%2FTNandFDT&mode=list