Practical Study of Deterministic Regular Expressions from Large-scale XML and Schema Data

TL;DR

This study analyzes the prevalence and subclasses of deterministic regular expressions (DREs) in large-scale real-world XML and schema datasets, revealing their widespread use and highlighting the need for further subclass research.

Contribution

First large-scale empirical analysis of DRE usage in Relax NG and RegExLib datasets, introducing a new DRE dataset and schema referencing analysis.

Findings

Over 98% of Relax NG expressions are DRE

Over 56% of RegExLib expressions are DRE

DRE subclasses require further study

Abstract

Regular expressions are a fundamental concept in computer science and widely used in various applications. In this paper we focused on deterministic regular expressions (DREs). Considering that researchers didn't have large datasets as evidence before, we first harvested a large corpus of real data from the Web then conducted a practical study to investigate the usage of DREs. One feature of our work is that the data set is sufficiently large compared with previous work, which is obtained using several data collection strategies we proposed. The results show more than 98\% of expressions in Relax NG are DRE, and more than 56\% of expressions from RegExLib are DRE, while both Relax NG and RegExLib do not have the determinism constraint. These observations indicate that DREs are commonly used in practice. The results also show further study of subclasses of DREs is necessary. As far as we know, we are the first to analyze the determinism and the subclasses of DREs of Relax NG and RegExLib, and give these results. Furthermore, we give some discussions and applications of the data set. We obtain a DRE data set from the original data, which will be useful in practice and it has value in its own right. We find current research in new subclasses of DREs is insufficient, therefore it is necessary to do further study. We also analyze the referencing relationships among XSDs and define SchemaRank, which can be used in XML Schema design.