Reason Analogically via Cross-domain Prior Knowledge: An Empirical Study of Cross-domain Knowledge Transfer for In-Context Learning

TL;DR

This paper empirically investigates how cross-domain knowledge transfer can enhance in-context learning by leveraging shared reasoning structures, demonstrating positive transfer under certain conditions and emphasizing the importance of retrieval methods.

Contribution

It provides the first comprehensive empirical validation of cross-domain knowledge transfer in in-context learning, highlighting the role of reasoning structure repair and retrieval strategies.

Findings

Conditional positive transfer observed in cross-domain ICL

Identification of an absorption threshold for effective transfer

Retrieval-based reasoning structure repair improves ICL performance

Abstract

Despite its success, existing in-context learning (ICL) relies on in-domain expert demonstrations, limiting its applicability when expert annotations are scarce. We posit that different domains may share underlying reasoning structures, enabling source-domain demonstrations to improve target-domain inference despite semantic mismatch. To test this hypothesis, we conduct a comprehensive empirical study of different retrieval methods to validate the feasibility of achieving cross-domain knowledge transfer under the in-context learning setting. Our results demonstrate conditional positive transfer in cross-domain ICL. We identify a clear example absorption threshold: beyond it, positive transfer becomes more likely, and additional demonstrations yield larger gains. Further analysis suggests that these gains stem from reasoning structure repair by retrieved cross-domain examples, rather than semantic cues. Overall, our study validates the feasibility of leveraging cross-domain knowledge transfer to improve cross-domain ICL performance, motivating the community to explore designing more effective retrieval approaches for this novel direction.\footnote{Our implementation is available at https://github.com/littlelaska/ICL-TF4LR}