Training for Technology: Adoption and Productive Use of Generative AI in Legal Analysis

TL;DR

This study shows that targeted training significantly improves law students' productive use of generative AI, increasing adoption, accuracy, and scores in legal analysis tasks.

Contribution

It demonstrates that training enhances the productive adoption of generative AI in professional settings, countering unproductive usage among untrained users.

Findings

Training increases AI adoption from 26% to 41%.

Trained users scored 0.27 points higher than untrained users.

Training shifts adoption patterns, especially among higher-ability students.

Abstract

Can targeted user training unlock the productive potential of generative artificial intelligence in professional settings? We study this question using a randomized experiment in which 164 law students completed an issue-spotting examination under one of three conditions: no GenAI access, optional access to a large language model (LLM), or LLM access with a brief training intervention. Untrained LLM access proved counterproductive: relative to participants without any LLM access, untrained users wrote significantly shorter answers, committed more case misstatements, and scored marginally lower, though most differences fall short of conventional significance. Training reversed this pattern. Trained participants adopted the LLM at higher rates (41% vs. 26%; p = 0.044), scored 0.27 grade points higher than untrained users--roughly one fine grade--(p = 0.027), and stated applicable rules more accurately (p = 0.014). Principal stratification analysis suggests training operates primarily through adoption rather than effectiveness--the adoption lower bound (1.06) exceeds the effectiveness upper bound (0.42) at strict mean dominance--though confidence intervals are wide. Training also shifted who adopted: top-quartile students went from 0% adoption to 42%. More broadly, these findings challenge the view that GenAI primarily benefits lower-skilled workers: without training, higher-ability practitioners opt out while lower-ability users adopt but unproductively. Realizing GenAI's productivity gains requires investment in both access and instruction.