The Effect of Pointer Analysis on Semantic Conflict Detection

TL;DR

This study empirically evaluates how pointer analysis impacts semantic conflict detection, revealing trade-offs between precision and recall that suggest hybrid approaches may be necessary.

Contribution

It provides the first empirical comparison of semantic conflict detection with and without pointer analysis, highlighting its effects on accuracy and performance.

Findings

Pointer analysis reduces false positives and timeouts.

Pointer analysis significantly decreases recall and F1-score.

Hybrid analysis techniques are recommended for better results.

Abstract

Current merge tools don't detect semantic conflicts, which occur when changes from different developers are textually integrated but semantically interfere with each other. Although researchers have proposed static analyses for detecting semantic conflicts, these analyses suffer from significant false positive rates. To understand whether such false positives could be reduced by using pointer analysis in the implementation of semantic conflict static analyses, we conduct an empirical study. We implement the same analysis with and without pointer analysis, run them on two datasets, observe how often they differ, and compare their accuracy and computational performance. Although pointer analysis is known to improve precision in static analysis, we find that its effect on semantic conflict detection can be drastic: we observe a significant reduction in timeouts and false positives, but also a significant increase in false negatives, with prohibitive drops in recall and F1-score. These results suggest that, in the context of semantic conflict detection, we should explore hybrid analysis techniques, combining aspects of both implementations we compare in our study.