ParsEval: Evaluation of Parsing Behavior using Real-world Out-in-the-wild X.509 Certificates

TL;DR

This paper evaluates the behavior of various X.509 certificate parsers using a large dataset of real-world certificates, revealing significant differences and an anomaly in wolfSSL's parser, highlighting ecosystem fragmentation.

Contribution

It introduces a comprehensive test suite and analysis of real-world X.509 parsers, exposing differences and anomalies in their handling of certificates.

Findings

Identified an anomaly in wolfSSL's parser

Revealed fundamental differences among parsers

Validated variability with real-world certificates

Abstract

X.509 certificates play a crucial role in establishing secure communication over the internet by enabling authentication and data integrity. Equipped with a rich feature set, the X.509 standard is defined by multiple, comprehensive ISO/IEC documents. Due to its internet-wide usage, there are different implementations in multiple programming languages leading to a large and fragmented ecosystem. This work addresses the research question "Are there user-visible and security-related differences between X.509 certificate parsers?". Relevant libraries offering APIs for parsing X.509 certificates were investigated and an appropriate test suite was developed. From 34 libraries 6 were chosen for further analysis. The X.509 parsing modules of the chosen libraries were called with 186,576,846 different certificates from a real-world dataset and the observed error codes were investigated. This study reveals an anomaly in wolfSSL's X.509 parsing module and that there are fundamental differences in the ecosystem. While related studies nowadays mostly focus on fuzzing techniques resulting in artificial certificates, this study confirms that available X.509 parsing modules differ largely and yield different results, even for real-world out-in-the-wild certificates.