The Impact of AI-Generated Solutions on Software Architecture and Productivity: Results from a Survey Study

TL;DR

This survey study investigates how AI-generated solutions influence software engineers' productivity and software quality, revealing significant productivity gains but challenges with complex projects and larger code solutions.

Contribution

It provides empirical insights into the effects of AI tools on software development productivity and quality, especially highlighting limitations with complex and large-scale problems.

Findings

AI tools significantly boost productivity for software engineers.

Productivity benefits decrease with increasing project complexity.

AI-generated solutions maintain quality for small snippets but decline for complex, larger problems.

Abstract

AI-powered software tools are widely used to assist software engineers. However, there is still a need to understand the productivity benefits of such tools for software engineers. In addition to short-term benefits, there is a question of how adopting AI-generated solutions affects the quality of software over time (e.g., maintainability and extendability). To provide some insight on these questions, we conducted a survey among software practitioners who use AI tools. Based on the data collected from our survey, we conclude that AI tools significantly increase the productivity of software engineers. However, the productivity benefits of using AI tools reduce as projects become more complex. The results also show that there are no significant negative influences of adopting AI-generated solutions on software quality, as long as those solutions are limited to smaller code snippets. However, when solving larger and more complex problems, AI tools generate solutions of a lower quality, indicating the need for architects to perform problem decomposition and solution integration.