DWMD: Dimensional Weighted Orderwise Moment Discrepancy for Domain-specific Hidden Representation Matching

TL;DR

This paper introduces DWMD, a novel high-order moment discrepancy metric for unsupervised domain adaptation, which explicitly aligns distribution differences in feature representations without distribution assumptions.

Contribution

The paper proposes DWMD, a theoretically validated, error-free, high-order moment discrepancy metric with dimensional weighting for improved domain-specific feature matching in UDA.

Findings

DWMD achieves state-of-the-art results on benchmark datasets.

The metric strictly reflects distribution differences without assumptions.

Empirical error bounds support practical applicability.

Abstract

Knowledge transfer from a source domain to a different but semantically related target domain has long been an important topic in the context of unsupervised domain adaptation (UDA). A key challenge in this field is establishing a metric that can exactly measure the data distribution discrepancy between two homogeneous domains and adopt it in distribution alignment, especially in the matching of feature representations in the hidden activation space. Existing distribution matching approaches can be interpreted as failing to either explicitly orderwise align higher-order moments or satisfy the prerequisite of certain assumptions in practical uses. We propose a novel moment-based probability distribution metric termed dimensional weighted orderwise moment discrepancy (DWMD) for feature representation matching in the UDA scenario. Our metric function takes advantage of a series for high-order moment alignment, and we theoretically prove that our DWMD metric function is error-free, which means that it can strictly reflect the distribution differences between domains and is valid without any feature distribution assumption. In addition, since the discrepancies between probability distributions in each feature dimension are different, dimensional weighting is considered in our function. We further calculate the error bound of the empirical estimate of the DWMD metric in practical applications. Comprehensive experiments on benchmark datasets illustrate that our method yields state-of-the-art distribution metrics.