Software engineering for artificial intelligence and machine learning software: A systematic literature review

TL;DR

This systematic review examines how software engineering practices are applied to AI/ML system development, highlighting current challenges, practices, and gaps in research from 1990 to 2019.

Contribution

It provides a comprehensive overview of the state of the art in applying software engineering to AI/ML, identifying key challenges and practices, and analyzing research limitations.

Findings

AI/ML systems are mainly developed in labs or large companies.

Main challenges include testing, quality, and data management.

Most practices are guidelines, lessons learned, or tools.

Abstract

Artificial Intelligence (AI) or Machine Learning (ML) systems have been widely adopted as value propositions by companies in all industries in order to create or extend the services and products they offer. However, developing AI/ML systems has presented several engineering problems that are different from those that arise in, non-AI/ML software development. This study aims to investigate how software engineering (SE) has been applied in the development of AI/ML systems and identify challenges and practices that are applicable and determine whether they meet the needs of professionals. Also, we assessed whether these SE practices apply to different contexts, and in which areas they may be applicable. We conducted a systematic review of literature from 1990 to 2019 to (i) understand and summarize the current state of the art in this field and (ii) analyze its limitations and open challenges that will drive future research. Our results show these systems are developed on a lab context or a large company and followed a research-driven development process. The main challenges faced by professionals are in areas of testing, AI software quality, and data management. The contribution types of most of the proposed SE practices are guidelines, lessons learned, and tools.