Attention Mechanism and Heuristic Approach: Context-Aware File Ranking Using Multi-Head Self-Attention

TL;DR

This paper introduces a context-aware file ranking method using Multi-Head Self-Attention to improve recall in change impact analysis, effectively modeling feature dependencies and enhancing expert-like reasoning.

Contribution

It applies Multi-Head Self-Attention as a post-processing step to refine deterministic scores, capturing contextual feature relationships and improving recall in impacted file identification.

Findings

Top-50 recall improved from 62-65% to 78-82%.

Achieved 80% recall at Top-50 files.

Expert validation score increased from 6.5/10 to 8.6/10.

Abstract

The identification and ranking of impacted files within software reposi-tories is a key challenge in change impact analysis. Existing deterministic approaches that combine heuristic signals, semantic similarity measures, and graph-based centrality metrics have demonstrated effectiveness in nar-rowing candidate search spaces, yet their recall plateaus. This limitation stems from the treatment of features as linearly independent contributors, ignoring contextual dependencies and relationships between metrics that characterize expert reasoning patterns. To address this limitation, we propose the application of Multi-Head Self-Attention as a post-deterministic scoring refinement mechanism. Our approach learns contextual weighting between features, dynamically adjust-ing importance levels per file based on relational behavior exhibited across candidate file sets. The attention mechanism produces context-aware adjustments that are additively combined with deterministic scores, pre-serving interpretability while enabling reasoning similar to that performed by experts when reviewing change surfaces. We focus on recall rather than precision, as false negatives (missing impacted files) are far more costly than false positives (irrelevant files that can be quickly dismissed during review). Empirical evaluation on 200 test cases demonstrates that the introduc-tion of self-attention improves Top-50 recall from approximately 62-65% to between 78-82% depending on repository complexity and structure, achiev-ing 80% recall at Top-50 files. Expert validation yields improvement from 6.5/10 to 8.6/10 in subjective accuracy alignment. This transformation bridges the reasoning capability gap between deterministic automation and expert judgment, improving recall in repository-aware effort estimation.