Bootstrap Generalization Ability from Loss Landscape Perspective

TL;DR

This paper introduces a novel approach to domain generalization in computer vision by analyzing the loss landscape, demonstrating its effectiveness through extensive experiments and achieving competitive results without domain-invariant methods.

Contribution

It pioneers the integration of loss landscape theory into domain generalization, exploring aspects like backbone, regularization, training paradigm, and learning rate.

Findings

Effective in improving generalization on unseen domains

Achieved 3rd place in ECCV 2022 NICO Challenge without domain-invariant methods

Validated through extensive ablation studies and visualizations

Abstract

Domain generalization aims to learn a model that can generalize well on the unseen test dataset, i.e., out-of-distribution data, which has different distribution from the training dataset. To address domain generalization in computer vision, we introduce the loss landscape theory into this field. Specifically, we bootstrap the generalization ability of the deep learning model from the loss landscape perspective in four aspects, including backbone, regularization, training paradigm, and learning rate. We verify the proposed theory on the NICO++, PACS, and VLCS datasets by doing extensive ablation studies as well as visualizations. In addition, we apply this theory in the ECCV 2022 NICO Challenge1 and achieve the 3rd place without using any domain invariant methods.