The Impact of AI Usage and Informativeness on Skill Development in Logical Reasoning

TL;DR

This study investigates how AI usage and informativeness influence skill development in logical reasoning, revealing that AI can both hinder or help learning depending on its informativeness and usage patterns.

Contribution

It provides empirical evidence on the nuanced effects of AI assistance and informativeness on individual skill development in logical reasoning tasks.

Findings

Heavy AI usage correlates with weaker skill development.

High-information AI improves short-term performance without harming long-term learning.

AI informativeness mediates the impact of AI assistance on skill development.

Abstract

Artificial intelligence (AI) is being increasingly integrated into human problem-solving, yet its effects on individual skill development remain unclear. We examine how both AI usage and informativeness can shape learning in the context of a controlled logical reasoning task with on-demand access to AI assistance. We find that greater AI usage is associated with weaker skill development: heavy AI users underperform relative to comparable peers, whereas light AI users perform similarly to matched users who do not use AI. We also find in our study that these patterns are mediated by AI informativeness. Low-information AI neither improves immediate performance nor preserves performance after AI assistance is removed, and is linked to weaker learning overall. On the other hand, high-information AI was found to improve short-run performance without reducing post-AI outcomes on average in our experiments, but with heterogeneous effects. Our findings in general suggest that AI can, depending on context, either complement human skill development by amplifying independent reasoning or can act as a substitute that undermines such reasoning, with the implication that regulating AI access and usage will be important for promoting skill development in the presence of AI assistance.