On Autopilot? An Empirical Study of Human-AI Teaming and Review Practices in Open Source

TL;DR

This study examines how developers interact with AI-assisted pull requests in open source projects, revealing distinct patterns such as faster merging and minimal review for AI co-authored PRs, and highlights the lack of guidelines for AI usage.

Contribution

It provides the first detailed analysis of project-level guidelines and developer interaction patterns with AI-assisted pull requests in open source software.

Findings

AI co-authored PRs often originate from contributors without prior code ownership.

AI co-authored PRs are merged faster with minimal feedback.

Most repositories lack guidelines for AI-assisted coding practices.

Abstract

Large Language Models (LLMs) increasingly automate software engineering tasks. While recent studies highlight the accelerated adoption of ``AI as a teammate'' in Open Source Software (OSS), developer interaction patterns remain under-explored. In this work, we investigated project-level guidelines and developers' interactions with AI-assisted pull requests (PRs) by expanding the AIDev dataset to include finer-grained contributor code ownership and a comparative baseline of human-created PRs. We found that over 67.5\% of AI-co-authored PRs originate from contributors without prior code ownership. Despite this, the majority of repositories lack guidelines for AI-coding agent usage. Notably, we observed a distinct interaction pattern: AI-co-authored PRs are merged significantly faster with minimal feedback. In contrast to human-created PRs where non-owner developers receive the most feedback, AI-co-authored PRs from non-owners receive the least, with approximately 80\% merged without any explicit review. Finally, we discuss implications for developers and researchers.