EyeLayer: Integrating Human Attention Patterns into LLM-Based Code Summarization

TL;DR

EyeLayer is a novel module that integrates human eye-gaze data into large language models to improve code summarization accuracy, demonstrating significant performance gains across multiple models without disrupting existing representations.

Contribution

The paper introduces EyeLayer, a lightweight attention-augmentation module that incorporates human eye-gaze patterns into LLMs for enhanced code summarization.

Findings

EyeLayer outperforms fine-tuning baselines by up to 13.17% on BLEU-4.

It effectively models human attention patterns to improve semantic focus.

The approach generalizes across different LLM architectures.

Abstract

Code summarization is the task of generating natural language descriptions of source code, which is critical for software comprehension and maintenance. While large language models (LLMs) have achieved remarkable progress on this task, an open question remains: can human expertise in code understanding further guide and enhance these models? We propose EyeLayer, a lightweight attention-augmentation module that incorporates human eye-gaze patterns, as a proxy of human expertise, into LLM-based code summarization. EyeLayer models human attention during code reading via a Multimodal Gaussian Mixture, redistributing token embeddings based on learned parameters (\mu_i, \sigma_i^2) that capture where and how intensively developers focus. This design enables learning generalizable attention priors from eye-tracking data and incorporating them into LLMs seamlessly, without disturbing existing representations. We evaluate EyeLayer across diverse model families (i.e., LLaMA-3.2, Qwen3, and CodeBERT) covering different scales and architectures. EyeLayer consistently outperforms strong fine-tuning baselines across standard metrics, achieving gains of up to 13.17% on BLEU-4. These results demonstrate that human gaze patterns encode complementary attention signals that enhance the semantic focus of LLMs and transfer effectively across diverse models for code summarization.