From Helpful to Trustworthy: LLM Agents for Pair Programming

TL;DR

This research investigates how multi-agent LLM pair programming workflows can be structured to improve trustworthiness, reliability, and maintainability of generated software artifacts through systematic studies and iterative validation.

Contribution

It proposes a systematic approach to externalize developer intent and validate outputs in multi-agent LLM pair programming, addressing trust and reliability issues.

Findings

Externalizing intent improves artifact reliability.

Automated feedback enhances code validation.

Workflow design influences trust in LLM-based programming.

Abstract

LLM-based coding agents are increasingly used to generate code, tests, and documentation. Still, their outputs can be plausible yet misaligned with developer intent and provide limited evidence for review in evolving projects. This limits our understanding of how to structure LLM pair-programming workflows so that artifacts remain reliable, auditable, and maintainable over time. To address this gap, this doctoral research proposes a systematic study of multi-agent LLM pair programming that externalizes intent and uses development tools for iterative validation. The plan includes three studies: translating informal problem statements into standards aligned requirements and formal specifications; refining tests and implementations using automated feedback, such as solver-backed counterexamples; and supporting maintenance tasks, including refactoring, API migrations, and documentation updates, while preserving validated behavior. The expected outcome is a clearer understanding of when multi-agent workflows increase trust, along with practical guidance for building reliable programming assistants for real-world development.