Adversarial Lagrangian Integrated Contrastive Embedding for Limited Size Datasets

TL;DR

This paper introduces ALICE, a novel adversarial contrastive embedding method that enhances representation quality and generalization for small datasets through adversarial transfer, augmentation, and multi-objective regularization.

Contribution

The paper proposes a new ALICE framework combining adversarial transfer, contrastive learning, and Lagrangian regularization to improve small dataset embedding quality.

Findings

Improved accuracy and convergence on small datasets.

Enhanced feature representation with adversarial augmentation.

Effective regularization for low-rank and sparse embeddings.

Abstract

Certain datasets contain a limited number of samples with highly various styles and complex structures. This study presents a novel adversarial Lagrangian integrated contrastive embedding (ALICE) method for small-sized datasets. First, the accuracy improvement and training convergence of the proposed pre-trained adversarial transfer are shown on various subsets of datasets with few samples. Second, a novel adversarial integrated contrastive model using various augmentation techniques is investigated. The proposed structure considers the input samples with different appearances and generates a superior representation with adversarial transfer contrastive training. Finally, multi-objective augmented Lagrangian multipliers encourage the low-rank and sparsity of the presented adversarial contrastive embedding to adaptively estimate the coefficients of the regularizers automatically to the optimum weights. The sparsity constraint suppresses less representative elements in the feature space. The low-rank constraint eliminates trivial and redundant components and enables superior generalization. The performance of the proposed model is verified by conducting ablation studies by using benchmark datasets for scenarios with small data samples.