Conflict-aware Inference of Python Compatible Runtime Environments with Domain Knowledge Graph

TL;DR

PyCRE is a novel method that uses a domain knowledge graph and heuristic algorithms to accurately infer compatible Python runtime environments, significantly improving dependency resolution success rates.

Contribution

This paper introduces PyCRE, which leverages a domain-specific knowledge graph and heuristic dependency solving to enhance Python environment inference accuracy.

Findings

PyCRE resolves nearly twice as many import errors as previous methods.

PyCRE constructs knowledge graphs for over 10,000 Python packages.

PyCRE efficiently guarantees package compatibility through heuristic graph traversal.

Abstract

Code sharing and reuse is a widespread use practice in software engineering. Although a vast amount of open-source Python code is accessible on many online platforms, programmers often find it difficult to restore a successful runtime environment. Previous studies validated automatic inference of Python dependencies using pre-built knowledge bases. However, these studies do not cover sufficient knowledge to accurately match the Python code and also ignore the potential conflicts between their inferred dependencies, thus resulting in a low success rate of inference. In this paper, we propose PyCRE, a new approach to automatically inferring Python compatible runtime environments with domain knowledge graph (KG). Specifically, we design a domain-specific ontology for Python third-party packages and construct KGs for over 10,000 popular packages in Python 2 and Python 3. PyCRE discovers candidate libraries by measuring the matching degree between the known libraries and the third-party resources used in target code. For the NP-complete problem of dependency solving, we propose a heuristic graph traversal algorithm to efficiently guarantee the compatibility between packages. PyCRE achieves superior performance on a real-world dataset and efficiently resolves nearly half more import errors than previous methods.