Order-preserving Consistency Regularization for Domain Adaptation and Generalization

TL;DR

This paper introduces Order-preserving Consistency Regularization (OCR), a novel approach that enhances model robustness in cross-domain tasks by maintaining the order of classification probabilities, outperforming existing methods.

Contribution

The paper proposes OCR, a new regularization technique that preserves the order of predictions to improve domain adaptation and generalization in deep learning models.

Findings

Achieves superior performance on five cross-domain tasks.

Enhances model robustness to domain-specific variations.

Outperforms existing regularization methods.

Abstract

Deep learning models fail on cross-domain challenges if the model is oversensitive to domain-specific attributes, e.g., lightning, background, camera angle, etc. To alleviate this problem, data augmentation coupled with consistency regularization are commonly adopted to make the model less sensitive to domain-specific attributes. Consistency regularization enforces the model to output the same representation or prediction for two views of one image. These constraints, however, are either too strict or not order-preserving for the classification probabilities. In this work, we propose the Order-preserving Consistency Regularization (OCR) for cross-domain tasks. The order-preserving property for the prediction makes the model robust to task-irrelevant transformations. As a result, the model becomes less sensitive to the domain-specific attributes. The comprehensive experiments show that our method achieves clear advantages on five different cross-domain tasks.