SBOM Generation Tools in the Python Ecosystem: an In-Detail Analysis

TL;DR

This paper analyzes the effectiveness and limitations of SBOM generation tools within the Python ecosystem, highlighting issues with dependency data accuracy and standardization that impact security and transparency.

Contribution

It provides an in-depth analysis of four popular SBOM tools, identifying key issues affecting SBOM completeness and correctness in the PyPI ecosystem.

Findings

Dependency version inaccuracies affect SBOM reliability

Metadata inconsistencies hinder complete SBOM generation

Lack of standardization in PyPI metadata impacts tool effectiveness

Abstract

Software Bills of Material (SBOMs), which improve transparency by listing the components constituting software, are a key countermeasure to the mounting problem of Software Supply Chain attacks. SBOM generation tools take project source files and provide an SBOM as output, interacting with the software ecosystem. While SBOMs are a substantial improvement for security practitioners, providing a complete and correct SBOM is still an open problem. This paper investigates the causes of the issues affecting SBOM completeness and correctness, focusing on the PyPI ecosystem. We analyze four popular SBOM generation tools using the CycloneDX standard. Our analysis highlights issues related to dependency versions, metadata files, remote dependencies, and optional dependencies. Additionally, we identified a systematic issue with the lack of standards for metadata in the PyPI ecosystem. This includes inconsistencies in the presence of metadata files as well as variations in how their content is formatted.