Adapting, Fast and Slow: Transportable Circuits for Few-Shot Learning

TL;DR

This paper introduces a novel approach for zero-shot and few-shot learning across domains using transportable circuits based on causal graphs, enabling effective generalization with limited target data.

Contribution

It proposes Circuit-TR, a method leveraging causal transportability theory for domain adaptation and zero-shot generalization, with theoretical analysis and practical algorithms.

Findings

Theoretical characterization of few-shot learnability via circuit transportability.

Circuit-TR effectively adapts to new domains with limited data.

Simulations validate the theoretical insights and method effectiveness.

Abstract

Generalization across the domains is not possible without asserting a structure that constrains the unseen target domain w.r.t. the source domain. Building on causal transportability theory, we design an algorithm for zero-shot compositional generalization which relies on access to qualitative domain knowledge in form of a causal graph for intra-domain structure and discrepancies oracle for inter-domain mechanism sharing. \textit{Circuit-TR} learns a collection of modules (i.e., local predictors) from the source data, and transport/compose them to obtain a circuit for prediction in the target domain if the causal structure licenses. Furthermore, circuit transportability enables us to design a supervised domain adaptation scheme that operates without access to an explicit causal structure, and instead uses limited target data. Our theoretical results characterize classes of few-shot learnable tasks in terms of graphical circuit transportability criteria, and connects few-shot generalizability with the established notion of circuit size complexity; controlled simulations corroborate our theoretical results.