WhyFlow: Interrogative Debugger for Sensemaking Taint Analysis

TL;DR

WhyFlow introduces an interactive question-answer style debugging tool for taint analysis, enabling end-users to better understand data flows, improve accuracy, and reduce mental effort during security analysis.

Contribution

It is the first end-user debugging interface for taint analysis that supports interactive questions and speculative analysis, enhancing sensemaking capabilities.

Findings

Participants using WhyFlow achieved 21% higher accuracy.

Users reported a 45% reduction in mental demand.

WhyFlow improved confidence in identifying relevant flows.

Abstract

Taint analysis is a security analysis technique used to track the flow of potentially dangerous data through an application and its dependent libraries. Investigating why certain unexpected flows appear and why expected flows are missing is an important sensemaking process during end-user taint analysis. Existing taint analysis tools often do not provide this end-user debugging capability, where developers can ask why, why-not, and what-if questions about dataflows and reason about the impact of configuring sources and sinks, and models of third-party libraries that abstract permissible and impermissible data flows. Furthermore, the tree-view or list-view used in existing taint analyzer visualizations makes it difficult to reason about the global impact on connectivity between multiple sources and sinks. Inspired by the insight that sensemaking tool-generated results can be significantly improved by a QA inquiry process, we propose WhyFlow, the first end-user question-answer style debugging interface for taint analysis. It enables a user to ask why, why-not, and what-if questions to investigate the existence of suspicious flows, the non-existence of expected flows, and the global impact of third-party library models. WhyFlow performs speculative what-if analysis, to help a user in debugging how different connectivity assumptions affect overall results. A user study with 12 participants shows that participants using WhyFlow achieved 21% higher accuracy on average, compared to CodeQL. They also reported a 45% reduction in mental demand (NASA-TLX) and rated higher confidence in identifying relevant flows using WhyFlow.