Detecting Cryptographically Relevant Software Packages with Collaborative LLMs

TL;DR

This paper investigates using collaborative large language models as a privacy-preserving, efficient heuristic for identifying cryptographically relevant software packages in large IT environments, aiding the transition to post-quantum cryptography.

Contribution

It introduces a novel collaborative LLM framework for cryptographic asset discovery that operates on-premises, reducing manual effort and enhancing detection reliability.

Findings

LLM ensembles effectively filter cryptographic software.

On-premises LLMs offer privacy advantages over online models.

The approach reduces manual workload in cryptographic asset identification.

Abstract

IT systems are facing an increasing number of security threats, including advanced persistent attacks and future quantum-computing vulnerabilities. The move towards crypto-agility and post-quantum cryptography (PQC) requires a reliable inventory of cryptographic assets across heterogeneous IT environments. Due to the sheer amount of packets, it is infeasible to manually detect cryptographically relevant software. Further, static code analysis pipelines often fail to address the diversity of modern ecosystems. Our research explores the use of large language models (LLMs) as heuristic tools for cryptographic asset discovery. We propose a collaborative framework that employs multiple LLMs to assess software relevance and aggregates their outputs through majority voting. To preserve data privacy, the approach operates on-premises without reliance on external servers. Using over 65,000 Fedora Linux packages, we evaluate the reliability of this method through statistical analysis, inter-model agreement, and manual validation. Preliminary results suggest that~LLM ensembles can serve as an efficient first-pass filter for identifying cryptographic software, resulting in reduced manual workload and assisting PQC transition. The study also compares on-premises and online LLM configurations, highlighting key advantages, limitations, and future directions for automated cryptographic asset discovery.