
TL;DR
This paper formalizes the concept of data validation, enabling clearer communication, automation, and analysis of data quality requirements by classifying them into increasing levels of complexity.
Contribution
It introduces a formal definition of data validation and demonstrates how to classify and analyze data quality requirements systematically.
Findings
Formal definition of data validation provided
Classification of data quality requirements into complexity levels
Insights into combining multiple validation requirements
Abstract
Data validation is the activity where one decides whether or not a particular data set is fit for a given purpose. Formalizing the requirements that drive this decision process allows for unambiguous communication of the requirements, automation of the decision process, and opens up ways to maintain and investigate the decision process itself. The purpose of this article is to formalize the definition of data validation and to demonstrate some of the properties that can be derived from this definition. In particular, it is shown how a formal view of the concept permits a classification of data quality requirements, allowing them to be ordered in increasing levels of complexity. Some subtleties arising from combining possibly many such requirements are pointed out as well.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
