From Noise to Knowledge: Interactive Summaries for Developer Alerts

TL;DR

CLARITY is an interactive tool that helps developers understand bug warnings more efficiently by summarizing and grouping related warnings through active feedback, improving the sensemaking process.

Contribution

This paper introduces CLARITY, a novel interactive summarization system that infers warning groupings based on user feedback to enhance bug report analysis.

Findings

Users identified root causes faster with CLARITY.

Significant individual variation in preferred groupings.

Few interactions needed to align rules with user labels.

Abstract

Programmers using bug-finding tools often review their reported warnings one by one. Based on the insight that identifying recurring themes and relationships can enhance the cognitive process of sensemaking, we propose CLARITY, which supports interpreting tool-generated warnings through interactive inquiry. CLARITY derives summary rules for custom grouping of related warnings with active feedback. As users mark warnings as interesting or uninteresting, CLARITY's rule inference algorithm surfaces common symptoms, highlighting structural similarities in containment, subtyping, invoked methods, accessed fields, and expressions. We demonstrate CLARITY on Infer and SpotBugs warnings across two mature Java projects. In a within-subject user study with 14 participants, users articulated root causes for similar uninteresting warnings faster and with more confidence using CLARITY. We observed significant individual variation in desired grouping, reinforcing the need for customizable sensemaking. Simulation shows that with rule-level feedback, only 11.8 interactions are needed on average to align all inferred rules with a simulated user's labels (vs. 17.8 without). Our evaluation suggests that CLARITY's active learning-based summarization enhances interactive warning sensemaking.