Information-Theoretic Detection of Unusual Source Code Changes

TL;DR

This paper introduces an information-theoretic approach to measure source code evolution and detect unusual changes, using entropy of tokens and syntax trees, showing promising results in anomaly detection.

Contribution

It presents a novel entropy-based method for analyzing code complexity and identifying unusual changes, expanding beyond traditional metrics.

Findings

Entropy correlates with code complexity measures.

Entropy-based anomaly detection achieves over 60% precision.

Entropy captures different complexity dimensions than classic metrics.

Abstract

The code base of software projects evolves essentially through inserting and removing information to and from the source code. We can measure this evolution via the elements of information - tokens, words, nodes - of the respective representation of the code. In this work, we approach the measurement of the information content of the source code of open-source projects from an information-theoretic standpoint. Our focus is on the entropy of two fundamental representations of code: tokens and abstract syntax tree nodes, from which we derive definitions of textual and structural entropy. We proceed with an empirical assessment where we evaluate the evolution patterns of the entropy of 95 actively maintained open source projects. We calculate the statistical relationships between our derived entropy metrics and classic methods of measuring code complexity and learn that entropy may capture different dimensions of complexity than classic metrics. Finally, we conduct entropy-based anomaly detection of unusual changes to demonstrate that our approach may effectively recognise unusual source code change events with over 60% precision, and lay the groundwork for improvements to information-theoretic measurement of source code evolution, thus paving the way for a new approach to statically gauging program complexity throughout its development.