A statistical approach for finding property-access errors

TL;DR

This paper presents a statistical method to detect property-access errors in JavaScript by analyzing real-world code patterns and filtering out safe anomalies, achieving high precision and recall in identifying actual bugs.

Contribution

It introduces a two-phase approach combining statistical analysis and data-flow filtering to effectively identify property-access errors in dynamic JavaScript objects.

Findings

Achieves 82% precision and 90% recall in detecting property-access errors.

VSCode code completion has 100% precision but only 22.5% recall for suggesting correct properties.

The approach is practical and suitable for real-world debugging.

Abstract

We study the problem of finding incorrect property accesses in JavaScript where objects do not have a fixed layout, and properties (including methods) can be added, overwritten, and deleted freely throughout the lifetime of an object. Since referencing a non-existent property is not an error in JavaScript, accidental accesses to non-existent properties (caused, perhaps, by a typo or by a misunderstanding of API documentation) can go undetected without thorough testing, and may manifest far from the source of the problem. We propose a two-phase approach for detecting property access errors based on the observation that, in practice, most property accesses will be correct. First a large number of property access patterns is collected from an extensive corpus of real-world JavaScript code, and a statistical analysis is performed to identify anomalous usage patterns. Specific instances of these patterns may not be bugs (due, e.g., dynamic type checks), so a local data-flow analysis filters out instances of anomalous property accesses that are safe and leaves only those likely to be actual bugs. We experimentally validate our approach, showing that on a set of 100 concrete instances of anomalous property accesses, the approach achieves a precision of 82% with a recall of 90%, making it suitable for practical use. We also conducted an experiment to determine how effective the popular VSCode code completion feature is at suggesting object properties, and found that, while it never suggested an incorrect property (precision of 100%), it failed to suggest the correct property in 62 out of 80 cases (recall of 22.5%). This shows that developers cannot rely on VSCode's code completion alone to ensure that all property accesses are valid.