Class-Level Logit Perturbation
Mengyang Li, Fengguang Su, Ou Wu, Ji Zhang

TL;DR
This paper explores class-level logit perturbation in neural networks, providing a theoretical foundation, new learning methodologies, and demonstrating competitive results on image classification benchmarks, including long-tail datasets.
Contribution
It introduces explicit logit perturbation methods for classification, unifies existing data augmentation and loss variation perspectives, and offers a plug-in approach adaptable to various algorithms.
Findings
Effective in improving model robustness and generalization.
Competitive performance on standard and long-tail datasets.
Applicable as a plug-in to existing classification methods.
Abstract
Features, logits, and labels are the three primary data when a sample passes through a deep neural network. Feature perturbation and label perturbation receive increasing attention in recent years. They have been proven to be useful in various deep learning approaches. For example, (adversarial) feature perturbation can improve the robustness or even generalization capability of learned models. However, limited studies have explicitly explored for the perturbation of logit vectors. This work discusses several existing methods related to class-level logit perturbation. A unified viewpoint between positive/negative data augmentation and loss variations incurred by logit perturbation is established. A theoretical analysis is provided to illuminate why class-level logit perturbation is useful. Accordingly, new methodologies are proposed to explicitly learn to perturb logits for both…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Data Classification · Anomaly Detection Techniques and Applications · Adversarial Robustness in Machine Learning
