Neural Network Embeddings for Test Case Prioritization

TL;DR

This paper introduces NNE-TCP, a machine learning framework that uses neural network embeddings to prioritize test cases in continuous integration, improving fault detection efficiency and enabling visualization of test-file relationships.

Contribution

The paper presents a novel ML-based tool that learns relationships between files and tests via embeddings, enhancing test prioritization and visualization in software testing.

Findings

NNE-TCP effectively prioritizes relevant tests based on file modifications.

The approach outperforms traditional test prioritization methods.

Embedding-based visualization reveals meaningful groupings of files and tests.

Abstract

In modern software engineering, Continuous Integration (CI) has become an indispensable step towards systematically managing the life cycles of software development. Large companies struggle with keeping the pipeline updated and operational, in useful time, due to the large amount of changes and addition of features, that build on top of each other and have several developers, working on different platforms. Associated with such software changes, there is always a strong component of Testing. As teams and projects grow, exhaustive testing quickly becomes inhibitive, becoming adamant to select the most relevant test cases earlier, without compromising software quality. We have developed a new tool called Neural Network Embeeding for Test Case Prioritization (NNE-TCP) is a novel Machine-Learning (ML) framework that analyses which files were modified when there was a test status transition and learns relationships between these files and tests by mapping them into multidimensional vectors and grouping them by similarity. When new changes are made, tests that are more likely to be linked to the files modified are prioritized, reducing the resources needed to find newly introduced faults. Furthermore, NNE-TCP enables entity visualization in low-dimensional space, allowing for other manners of grouping files and tests by similarity and to reduce redundancies. By applying NNE-TCP, we show for the first time that the connection between modified files and tests is relevant and competitive relative to other traditional methods.