Developer Needs and Feasible Features for AI Assistants in IDEs

TL;DR

This study identifies developer needs for AI assistants in IDEs through interviews and feasibility analysis, highlighting gaps in current features and guiding future development for more effective AI support.

Contribution

It combines user interviews and prediction markets to uncover developer needs and assess feature feasibility, revealing gaps and directions for AI assistant development in IDEs.

Findings

Developers prioritize features on implementation and context awareness.

Proactivity and maintenance features are underestimated and underdeveloped.

Significant gaps exist between developer needs and current AI support in IDEs.

Abstract

Despite the increasing presence of AI assistants in Integrated Development Environments (IDEs), it remains unclear what different groups of developers actually need from these tools and which features are likely to be implemented in practice. To investigate this gap, we conducted a two-phase study. First, we interviewed 35 professional developers from three user groups (Adopters, Churners, and Non-Users) to uncover unmet needs and expectations. Our analysis revealed five key areas of need distinctly distributed across practitioners' groups: Technology Improvement, Interaction, and Customization, as well as Simplifying Skill Building, and Programming Tasks. We then examined the feasibility of addressing selected needs through an internal prediction market involving 102 practitioners. The results demonstrate a strong alignment between the developers' needs and the practitioners' judgment for features focused on implementation and context awareness. However, features related to proactivity and maintenance remain both underestimated and technically unaddressed. Our findings reveal gaps in current AI support and provide practical directions for developing more effective and sustainable in-IDE AI systems