Case Study: Using AI-Assisted Code Generation In Mobile Teams

TL;DR

This case study evaluates the impact of AI-assisted code generation on mobile development teams, focusing on onboarding and technology switching in Kotlin and Swift, measuring performance and integration.

Contribution

It provides empirical evidence on how AI tools affect onboarding and transition efficiency in native mobile development teams.

Findings

AI-assisted coding improved onboarding speed

Enhanced correctness in code solutions with AI support

Positive developer feedback on AI integration

Abstract

The aim of this study is to evaluate the performance of AI-assisted programming in actual mobile development teams that are focused on native mobile languages like Kotlin and Swift. The extensive case study involves 16 participants and 2 technical reviewers, from a software development department designed to understand the impact of using LLMs trained for code generation in specific phases of the team, more specifically, technical onboarding and technical stack switch. The study uses technical problems dedicated to each phase and requests solutions from the participants with and without using AI-Code generators. It measures time, correctness, and technical integration using ReviewerScore, a metric specific to the paper and extracted from actual industry standards, the code reviewers of merge requests. The output is converted and analyzed together with feedback from the participants in an attempt to determine if using AI-assisted programming tools will have an impact on getting developers onboard in a project or helping them with a smooth transition between the two native development environments of mobile development, Android and iOS. The study was performed between May and June 2023 with members of the mobile department of a software development company based in Cluj-Napoca, with Romanian ownership and management.