When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration

TL;DR

This paper introduces KITE, a framework for evaluating human-AI knowledge transfer, revealing that high AI performance does not always translate to effective knowledge sharing, highlighting the need for dedicated optimization.

Contribution

The paper presents the KITE framework and a large-scale human study to measure and analyze knowledge transfer in human-AI collaboration, emphasizing the importance of optimizing for communication.

Findings

Model benchmark performance correlates with outcomes but has many outliers.

Knowledge transfer effectiveness varies significantly across different strategies.

Dedicated optimization is necessary for effective human-AI knowledge sharing.

Abstract

Recent advancements in AI reasoning have driven substantial improvements across diverse tasks. A critical open question is whether these improvements also yields better knowledge transfer: the ability of models to communicate reasoning in ways humans can understand, apply, and learn from. To investigate this, we introduce Knowledge Integration and Transfer Evaluation (KITE), a conceptual and experimental framework for Human-AI knowledge transfer capabilities and conduct the first large-scale human study (N=118) explicitly designed to measure it. In our two-phase setup, humans first ideate with an AI on problem-solving strategies, then independently implement solutions, isolating model explanations' influence on human understanding. Our findings reveal that although model benchmark performance correlates with collaborative outcomes, this relationship is notably inconsistent, featuring significant outliers, indicating that knowledge transfer requires dedicated optimization. Our analysis identifies behavioral and strategic factors mediating successful knowledge transfer. We release our code, dataset, and evaluation framework to support future work on communicatively aligned models.