A Critical Study of What Code-LLMs (Do Not) Learn

TL;DR

This study provides a detailed analysis of code-LLMs, revealing that they encode certain token relations but fail to encode others, especially between syntactic tokens and identifiers, with larger models encoding less information.

Contribution

It offers a novel fine-grained analysis of what code-LLMs encode, highlighting limitations in their understanding of code relations and effects of model size and fine-tuning.

Findings

Models encode relations among syntactic tokens and identifiers separately.

Models fail to encode relations between syntactic tokens and identifiers.

Larger models encode less information about code.

Abstract

Large Language Models trained on code corpora (code-LLMs) have demonstrated impressive performance in various coding assistance tasks. However, despite their increased size and training dataset, code-LLMs still have limitations such as suggesting codes with syntactic errors, variable misuse etc. Some studies argue that code-LLMs perform well on coding tasks because they use self-attention and hidden representations to encode relations among input tokens. However, previous works have not studied what code properties are not encoded by code-LLMs. In this paper, we conduct a fine-grained analysis of attention maps and hidden representations of code-LLMs. Our study indicates that code-LLMs only encode relations among specific subsets of input tokens. Specifically, by categorizing input tokens into syntactic tokens and identifiers, we found that models encode relations among syntactic tokens and among identifiers, but they fail to encode relations between syntactic tokens and identifiers. We also found that fine-tuned models encode these relations poorly compared to their pre-trained counterparts. Additionally, larger models with billions of parameters encode significantly less information about code than models with only a few hundred million parameters.