Modeling Programmer Attention as Scanpath Prediction

TL;DR

This paper introduces a new approach to modeling programmer attention by predicting eye movement scanpaths, aiming to improve interfaces and AI by understanding what programmers focus on during coding tasks.

Contribution

It presents the first attempt at scanpath prediction in programming, combining eye tracking data with a prototype model to advance understanding of programmer attention.

Findings

Preliminary results from 27 programmers' eye tracking data.

A prototype scanpath prediction model is developed.

Early community feedback obtained.

Abstract

This paper launches a new effort at modeling programmer attention by predicting eye movement scanpaths. Programmer attention refers to what information people intake when performing programming tasks. Models of programmer attention refer to machine prediction of what information is important to people. Models of programmer attention are important because they help researchers build better interfaces, assistive technologies, and more human-like AI. For many years, researchers in SE have built these models based on features such as mouse clicks, key logging, and IDE interactions. Yet the holy grail in this area is scanpath prediction -- the prediction of the sequence of eye fixations a person would take over a visual stimulus. A person's eye movements are considered the most concrete evidence that a person is taking in a piece of information. Scanpath prediction is a notoriously difficult problem, but we believe that the emergence of lower-cost, higher-accuracy eye tracking equipment and better large language models of source code brings a solution within grasp. We present an eye tracking experiment with 27 programmers and a prototype scanpath predictor to present preliminary results and obtain early community feedback.