Ver: View Discovery in the Wild

TL;DR

Ver is a data discovery system designed to identify project-join views over large, unannotated repositories, effectively handling scale, search, and semantic ambiguity issues, demonstrated through user studies and large-scale experiments.

Contribution

It introduces a novel architecture for view discovery that addresses both technical scalability and human semantic challenges in large datasets.

Findings

Users successfully find desired views using Ver

Ver performs efficiently on datasets with tens of millions of join paths

The system effectively handles inaccurate input queries

Abstract

We present Ver, a data discovery system that identifies project-join views over large repositories of tables that do not contain join path information, and even when input queries are inaccurate. Ver implements a reference architecture to solve both the technical (scale and search) and human (semantic ambiguity, navigating a large number of results) problems of view discovery. We demonstrate users find the view they want when using Ver with a user study and we demonstrate its performance with large-scale end-to-end experiments on real-world datasets containing tens of millions of join paths.