DAInfer+: Neurosymbolic Inference of API Specifications from Documentation via Embedding Models

TL;DR

DAInfer+ is a novel approach that uses NLP and neurosymbolic optimization to infer detailed API specifications from documentation, improving static analysis accuracy for software libraries.

Contribution

It introduces a neurosymbolic optimization method leveraging sentence embeddings and LLMs to accurately derive API data-flow and aliasing specifications from documentation.

Findings

Embedding models outperform LLMs in robustness and semantic detail.

Achieves over 82% recall and 85% precision for data-flow inference.

Provides rapid inference within seconds, demonstrating practical utility.

Abstract

Modern software systems heavily rely on various libraries, which require understanding the API semantics in static analysis. However, summarizing API semantics remains challenging due to complex implementations or unavailable library code. This paper presents DAInfer+, a novel approach for inferring API specifications from library documentation. We employ Natural Language Processing (NLP) to interpret informal semantic information provided by the documentation, which enables us to reduce the specification inference to an optimization problem. Specifically, we investigate the effectiveness of sentence embedding models and Large Language Models (LLMs) in deriving memory operation abstractions from API descriptions. These abstractions are used to retrieve data-flow and aliasing relations to generate comprehensive API specifications. To solve the optimization problem efficiently, we propose neurosymbolic optimization, yielding precise data-flow and aliasing specifications. Our evaluation of popular Java libraries shows that zero-shot sentence embedding models outperform few-shot prompted LLMs in robustness, capturing fine-grained semantic nuances more effectively. While our initial attempts using two-stage LLM prompting yielded promising results, we found that the embedding-based approach proved superior. Specifically, these models achieve over 82% recall and 85% precision for data-flow inference and 88% recall and 79% precision for alias relations, all within seconds. These results demonstrate the practical value of DAInfer+ in library-aware static analysis.