MUTE: Data-Similarity Driven Multi-hot Target Encoding for Neural Network Design

TL;DR

MUTE is a novel target encoding method that enhances neural network performance by understanding class similarities, improving robustness and generalization with minimal additional computation.

Contribution

The paper introduces MUTE, a data-similarity driven target encoding scheme that optimizes class relationships to improve neural network robustness and generalization without increasing model size.

Findings

MUTE improves classification accuracy on image datasets.

MUTE enhances robustness against noise and adversarial attacks.

MUTE achieves these benefits with negligible computational overhead.

Abstract

Target encoding is an effective technique to deliver better performance for conventional machine learning methods, and recently, for deep neural networks as well. However, the existing target encoding approaches require significant increase in the learning capacity, thus demand higher computation power and more training data. In this paper, we present a novel and efficient target encoding scheme, MUTE to improve both generalizability and robustness of a target model by understanding the inter-class characteristics of a target dataset. By extracting the confusion level between the target classes in a dataset, MUTE strategically optimizes the Hamming distances among target encoding. Such optimized target encoding offers higher classification strength for neural network models with negligible computation overhead and without increasing the model size. When MUTE is applied to the popular image classification networks and datasets, our experimental results show that MUTE offers better generalization and defense against the noises and adversarial attacks over the existing solutions.