Mining Type Constructs Using Patterns in AI-Generated Code

TL;DR

This paper empirically analyzes AI-generated TypeScript code, revealing AI's tendencies to misuse type constructs but also showing higher acceptance rates for AI-involved pull requests.

Contribution

First systematic empirical study comparing AI and human use of type constructs in TypeScript, highlighting AI's overuse of 'any' and advanced types.

Findings

AI agents are 9x more prone to use 'any' keyword.

AI agents use advanced type constructs more often than humans.

AI-involved pull requests have 1.8x higher acceptance rates.

Abstract

Artificial Intelligence (AI) increasingly automates various parts of the software development tasks. Although AI has enhanced the productivity of development tasks, it remains unstudied whether AI essentially outperforms humans in type-related programming tasks, such as employing type constructs properly for type safety, during its tasks. Moreover, there is no systematic study that evaluates whether AI agents overuse or misuse the type constructs under the complicated type systems to the same extent as humans. In this study, we present the first empirical analysis to answer these questions in the domain of TypeScript projects. Our findings show that, in contrast to humans, AI agents are 9x more prone to use the 'any' keyword. In addition, we observed that AI agents use advanced type constructs, including those that ignore type checks, more often compared to humans. Surprisingly, even with all these issues, Agentic pull requests (PRs) have 1.8x higher acceptance rates compared to humans for TypeScript. We encourage software developers to carefully confirm the type safety of their codebases whenever they coordinate with AI agents in the development process.