CV4Code: Sourcecode Understanding via Visual Code Representations

TL;DR

CV4Code introduces a novel image-based approach for source code understanding, enabling language-agnostic analysis and handling syntactically incorrect code, with state-of-the-art results in functional task prediction and code similarity retrieval.

Contribution

The paper presents a new image-based representation of source code that eliminates the need for language-specific parsing and tokenization, improving robustness and efficiency.

Findings

Achieves state-of-the-art performance in code classification and retrieval.

Handles syntactically incorrect code effectively.

Demonstrates the viability of treating source code as images for understanding tasks.

Abstract

We present CV4Code, a compact and effective computer vision method for sourcecode understanding. Our method leverages the contextual and the structural information available from the code snippet by treating each snippet as a two-dimensional image, which naturally encodes the context and retains the underlying structural information through an explicit spatial representation. To codify snippets as images, we propose an ASCII codepoint-based image representation that facilitates fast generation of sourcecode images and eliminates redundancy in the encoding that would arise from an RGB pixel representation. Furthermore, as sourcecode is treated as images, neither lexical analysis (tokenisation) nor syntax tree parsing is required, which makes the proposed method agnostic to any particular programming language and lightweight from the application pipeline point of view. CV4Code can even featurise syntactically incorrect code which is not possible from methods that depend on the Abstract Syntax Tree (AST). We demonstrate the effectiveness of CV4Code by learning Convolutional and Transformer networks to predict the functional task, i.e. the problem it solves, of the source code directly from its two-dimensional representation, and using an embedding from its latent space to derive a similarity score of two code snippets in a retrieval setup. Experimental results show that our approach achieves state-of-the-art performance in comparison to other methods with the same task and data configurations. For the first time we show the benefits of treating sourcecode understanding as a form of image processing task.