The Impact of SBOM Generators on Vulnerability Assessment in Python: A Comparison and a Novel Approach

TL;DR

This paper evaluates how SBOM generators affect vulnerability detection in Python, revealing limitations and introducing PIP-sbom, a new tool that significantly improves accuracy and reduces false positives in software supply chain security.

Contribution

The paper provides the first security analysis of SBOM tools' effectiveness in vulnerability detection and introduces PIP-sbom, a novel approach that enhances component accuracy and dependency resolution.

Findings

PIP-sbom increases precision and recall by 60% over existing tools.

PIP-sbom reduces false positives by ten times.

Current SBOM tools often produce inaccurate component and dependency data.

Abstract

The Software Supply Chain (SSC) security is a critical concern for both users and developers. Recent incidents, like the SolarWinds Orion compromise, proved the widespread impact resulting from the distribution of compromised software. The reliance on open-source components, which constitute a significant portion of modern software, further exacerbates this risk. To enhance SSC security, the Software Bill of Materials (SBOM) has been promoted as a tool to increase transparency and verifiability in software composition. However, despite its promise, SBOMs are not without limitations. Current SBOM generation tools often suffer from inaccuracies in identifying components and dependencies, leading to the creation of erroneous or incomplete representations of the SSC. Despite existing studies exposing these limitations, their impact on the vulnerability detection capabilities of security tools is still unknown. In this paper, we perform the first security analysis on the vulnerability detection capabilities of tools receiving SBOMs as input. We comprehensively evaluate SBOM generation tools by providing their outputs to vulnerability identification software. Based on our results, we identify the root causes of these tools' ineffectiveness and propose PIP-sbom, a novel pip-inspired solution that addresses their shortcomings. PIP-sbom provides improved accuracy in component identification and dependency resolution. Compared to best-performing state-of-the-art tools, PIP-sbom increases the average precision and recall by 60%, and reduces by ten times the number of false positives.