ATTest: Agent-Driven Tensor Testing for Deep Learning Library Modules

TL;DR

ATTest introduces an agent-driven framework for tensor testing in deep learning libraries, effectively addressing semantic challenges and outperforming existing methods in code coverage for PyTorch and TensorFlow.

Contribution

It presents a novel seven-stage pipeline with an iterative loop for constraint extraction, test generation, validation, and repair, improving test stability and coverage.

Findings

Achieves 55.60% branch coverage on PyTorch

Achieves 54.77% branch coverage on TensorFlow

Outperforms state-of-the-art baselines significantly

Abstract

The unit testing of Deep Learning (DL) libraries is challenging due to complex numerical semantics and implicit tensor constraints. Traditional Search-Based Software Testing (SBST) often suffers from semantic blindness, failing to satisfy the constraints of high-dimensional tensors, whereas Large Language Models (LLMs) struggle with cross-file context and unstable code modifications. This paper proposes ATTest, an agent-driven tensor testing framework for module-level unit test generation. ATTest orchestrates a seven-stage pipeline, which encompasses constraint extraction and an iterative "generation-validation-repair" loop, to maintain testing stability and mitigate context-window saturation. An evaluation on PyTorch and TensorFlow demonstrates that ATTest significantly outperforms state-of-the-art baselines such as PynguinML, achieving an average branch coverage of 55.60% and 54.77%, respectively. The results illustrate how agent-driven workflows bridge the semantic gap in numerical libraries while ensuring auditable test synthesis. Source code: https://github.com/iSEngLab/ATTest.git