Toward Neurosymbolic Program Comprehension

TL;DR

This paper advocates for a Neurosymbolic approach to program comprehension, combining deep learning models with symbolic methods to improve reliability, interpretability, and efficiency in software engineering tasks.

Contribution

It introduces the concept of Neurosymbolic Program Comprehension (NsPC), proposing a hybrid framework that integrates neural and symbolic techniques for better code analysis.

Findings

Preliminary results show promise for the hybrid approach.

The framework aims to enhance trustworthiness and interpretability.

Challenges in scaling large models motivate the Neurosymbolic direction.

Abstract

Recent advancements in Large Language Models (LLMs) have paved the way for Large Code Models (LCMs), enabling automation in complex software engineering tasks, such as code generation, software testing, and program comprehension, among others. Tools like GitHub Copilot and ChatGPT have shown substantial benefits in supporting developers across various practices. However, the ambition to scale these models to trillion-parameter sizes, exemplified by GPT-4, poses significant challenges that limit the usage of Artificial Intelligence (AI)-based systems powered by large Deep Learning (DL) models. These include rising computational demands for training and deployment and issues related to trustworthiness, bias, and interpretability. Such factors can make managing these models impractical for many organizations, while their "black-box'' nature undermines key aspects, including transparency and accountability. In this paper, we question the prevailing assumption that increasing model parameters is always the optimal path forward, provided there is sufficient new data to learn additional patterns. In particular, we advocate for a Neurosymbolic research direction that combines the strengths of existing DL techniques (e.g., LLMs) with traditional symbolic methods--renowned for their reliability, speed, and determinism. To this end, we outline the core features and present preliminary results for our envisioned approach, aimed at establishing the first Neurosymbolic Program Comprehension (NsPC) framework to aid in identifying defective code components.