[Re] Learning to Learn By Self-Critique

TL;DR

This reproducibility study confirms that the Self-Critique and Adapt (SCA) method enhances MAML++ performance, though some advanced versions remain unreproduced due to missing implementation details, highlighting reproducibility challenges.

Contribution

The paper validates the effectiveness of SCA in improving MAML++ and discusses reproducibility issues with complex variants due to omitted implementation details.

Findings

SCA improves MAML++ performance on few-shot learning tasks.

Reproduced results support the original claims about SCA.

Some advanced versions of MAML++ could not be reproduced.

Abstract

This work is a reproducibility study of the paper of Antoniou and Storkey [2019], published at NeurIPS 2019. Our results are in parts similar to the ones reported in the original paper, supporting the central claim of the paper that the proposed novel method, called Self-Critique and Adapt (SCA), improves the performance of MAML++. The conducted additional experiments on the Caltech-UCSD Birds 200 dataset confirm the superiority of SCA compared to MAML++. In addition, the reproduced paper suggests a novel high-end version of MAML++ for which we could not reproduce the same results. We hypothesize that this is due to the many implementation details that were omitted in the original paper.