You Don't Know Search: Helping Users Find Code by Automatically Evaluating Alternative Queries
Rijnard van Tonder

TL;DR
This paper introduces Automated Query Evaluation (AQE), a technique that automatically generates and tests alternative interpretations of ambiguous code search queries to improve user satisfaction and result engagement.
Contribution
The paper presents AQE, a novel method for disambiguating code search queries by automatically evaluating alternative interpretations, enhancing search effectiveness.
Findings
Users are 22% more likely to click on results with AQE active.
AQE reduces user frustration with ambiguous queries.
The approach is validated through an A/B test with over 10,000 users.
Abstract
Tens of thousands of engineers use Sourcegraph day-to-day to search for code and rely on it to make progress on software development tasks. We face a key challenge in designing a query language that accommodates the needs of a broad spectrum of users. Our experience shows that users express different and often contradictory preferences for how queries should be interpreted. These preferences stem from users with differing usage contexts, technical experience, and implicit expectations from using prior tools. At the same time, designing a code search query language poses unique challenges because it intersects traditional search engines and full-fledged programming languages. For example, code search queries adopt certain syntactic conventions in the interest of simplicity and terseness but invariably risk encoding implicit semantics that are ambiguous at face-value (a single space in a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Engineering Research · Open Source Software Innovations · Web Data Mining and Analysis
MethodsTest
