Accountable Agents in Software Engineering: An Analysis of Terms of Service and a Research Roadmap

TL;DR

This paper analyzes the Terms of Service of AI coding assistants to understand accountability issues and proposes a research roadmap for responsible autonomous agents in software engineering.

Contribution

It provides a comparative analysis of ToS documents, highlighting responsibility shifts and proposing a research agenda for accountability in AI-assisted software development.

Findings

Responsibility for correctness is often shifted to users.

Providers vary significantly in addressing liability and data reuse.

Existing policies are misaligned with autonomous software development workflows.

Abstract

AI coding assistants and autonomous agents are becoming integral to software development workflows, reshaping how code is produced, reviewed, and maintained. While recent research has focused mainly on the capabilities and impacts of productivity of these systems, much less attention has been paid to accountability: who is responsible when agents generate, modify, or recommend code? In practice, accountability is defined through the Terms of Service (ToS) and related policy documents that govern the use of AI-powered development tools. In this vision paper, we present a comparative analysis of the Terms of Service for widely used AI coding assistants and agent-enabled development tools. We examine how these documents allocate ownership, responsibility, liability, and disclosure obligations between tool providers and software developers, and we identify common patterns and divergences between providers. Our analysis reveals a consistent tendency to shift responsibility for correctness, safety, and legal compliance onto users, as well as substantial variation in how providers address issues such as indemnification, data reuse, and acceptable use. Based on these findings, we argue that existing policy frameworks are poorly aligned with increasingly agent-mediated and autonomous software development workflows. We outline a research roadmap for accountable agents in software engineering, identifying challenges and opportunities for modeling responsibility, designing governance artifacts, developing tooling that supports accountability, and conducting empirical studies of developers' perceptions and practices.