Human-AI Collaboration for Scaling Agile Regression Testing: An Agentic-AI Teammate from Manual to Automated Testing

TL;DR

This paper presents an agentic AI system that assists in regression testing within Agile teams, improving automation speed and code reuse while emphasizing the importance of human oversight.

Contribution

It introduces the Hacon Test Automation Copilot, an AI system that generates test scripts from specifications, integrating human review to enhance regression testing in Agile environments.

Findings

AI accelerates script authoring and increases throughput by 30-50%

Human review remains essential for maintainability and correctness

Explicit governance and collaboration are key for effective human-AI teaming

Abstract

Automated regression testing is essential for maintaining rapid, high-quality delivery in Agile and Scrum organizations. Many teams, including Hacon (a Siemens company), face a persistent gap: validated test specifications accumulate faster than they are automated, limiting regression coverage and increasing manual work. This paper reports an exploratory industrial case study of the Hacon Test Automation Copilot, an agentic AI system that generates system-level regression test scripts from validated specifications using retrieval-augmented generation and a multi-agent workflow. Integrated with Hacon's CI pipelines, the Copilot operates asynchronously as a "silent AI teammate", producing candidate scripts for human review. Mixed-method evaluation shows the AI accelerates script authoring and increases throughput, with 30-50% code reuse. However, human review remains necessary for maintainability and correct domain interpretation. Clear specifications, explicit governance, and ongoing human-AI collaboration are critical. We conclude with lessons for scaling regression automation and enabling effective human-AI teaming in Agile settings.