Knowledge-Guided Wasserstein Distributionally Robust Optimization

TL;DR

This paper introduces KG-WDRO, a transfer learning framework that adaptively incorporates external knowledge into Wasserstein DRO to improve statistical efficiency with small samples, reducing conservativeness and enhancing performance.

Contribution

The paper proposes a novel knowledge-guided Wasserstein DRO method that constructs smaller ambiguity sets using source knowledge, providing a new interpretation for shrinkage transfer learning approaches.

Findings

Outperforms existing methods in small-sample transfer learning scenarios.

Effectively incorporates multiple sources of external knowledge.

Demonstrates superior adaptivity and robustness through extensive simulations.

Abstract

Transfer learning is a popular strategy to leverage external knowledge and improve statistical efficiency, particularly with a limited target sample. We propose a novel knowledge-guided Wasserstein Distributionally Robust Optimization (KG-WDRO) framework that adaptively incorporates multiple sources of external knowledge to overcome the conservativeness of vanilla WDRO, which often results in overly pessimistic shrinkage toward zero. Our method constructs smaller Wasserstein ambiguity sets by controlling the transportation along directions informed by the source knowledge. This strategy can alleviate perturbations on the predictive projection of the covariates and protect against information loss. Theoretically, we establish the equivalence between our WDRO formulation and the knowledge-guided shrinkage estimation based on collinear similarity, ensuring tractability and geometrizing the feasible set. This also reveals a novel and general interpretation for recent shrinkage-based transfer learning approaches from the perspective of distributional robustness. In addition, our framework can adjust for scaling differences in the regression models between the source and target and accommodates general types of regularization such as lasso and ridge. Extensive simulations demonstrate the superior performance and adaptivity of KG-WDRO in enhancing small-sample transfer learning.