Fingerprinting AI Coding Agents on GitHub
Taher A. Ghaleb

TL;DR
This study analyzes behavioral signatures of AI coding agents on GitHub, achieving high accuracy in identifying which AI generated specific pull requests, revealing distinct patterns for different tools.
Contribution
First comprehensive fingerprinting of AI coding agents using 41 features across multiple tools with high identification accuracy.
Findings
97.2% F1-score in multi-class agent identification
Distinct behavioral signatures for different AI agents
AI-generated code exhibits detectable patterns
Abstract
AI coding agents are reshaping software development through both autonomous and human-mediated pull requests (PRs). When developers use AI agents to generate code under their own accounts, code authorship attribution becomes critical for repository governance, research validity, and understanding modern development practices. We present the first study on fingerprinting AI coding agents, analyzing 33,580 PRs from five major agents (OpenAI Codex, GitHub Copilot, Devin, Cursor, Claude Code) to identify behavioral signatures. With 41 features spanning commit messages, PR structure, and code characteristics, we achieve 97.2% F1-score in multi-class agent identification. We uncover distinct fingerprints: Codex shows unique multiline commit patterns (67.5% feature importance), and Claude Code exhibits distinctive code structure (27.2% importance of conditional statements). These signatures…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Engineering Research · Ethics and Social Impacts of AI · Scientific Computing and Data Management
