An Investigation into Maintenance Support for Neural Networks

TL;DR

This paper investigates current practices and challenges in maintaining neural networks, highlighting gaps in existing tools and methods, and emphasizing the need for improved maintenance support tailored to practitioners.

Contribution

It provides an empirical analysis of practitioners' approaches to neural network maintenance, revealing limitations of current tools and suggesting directions for future improvements.

Findings

Existing tools focus mainly on training, not maintenance.

Practitioners struggle with understanding and fixing unexpected behaviors.

There are significant gaps in current maintenance methodologies.

Abstract

As the potential for neural networks to augment our daily lives grows, ensuring their quality through effective testing, debugging, and maintenance is essential. This is especially the case as we acknowledge the prospects of negative impacts from these technologies. Traditional software engineering methods, such as testing and debugging, have proven effective in maintaining software quality; however, they reveal significant research and practice gaps in maintaining neural networks. In particular, there is a limited understanding of how practitioners currently address challenges related to understanding and mitigating undesirable behaviors in neural networks. In our ongoing research, we explore the current state of research and practice in maintaining neural networks by curating insights from practitioners through a preliminary study involving interviews and supporting survey responses. Our findings thus far indicate that existing tools primarily concentrate on building and training models. While these tools can be beneficial, they often fall short of supporting practitioners' understanding and addressing the underlying causes of unexpected model behavior. By evaluating current procedures and identifying the limitations of traditional methodologies, our study aims to offer a developer-centric perspective on where current practices fall short and highlight opportunities for improving maintenance support in neural networks.