A Datalake for Data-driven Social Science Research

TL;DR

This paper introduces a specialized Datalake infrastructure designed to facilitate data-driven social science research by integrating diverse data, ensuring transparency, and supporting analysis, thereby democratizing access to advanced data science tools.

Contribution

The paper presents a novel Datalake system tailored for social science research, supporting diverse data ingestion, provenance tracking, access control, and analysis tools, with real-world applications.

Findings

Streamlines social science research processes.

Enhances transparency and reproducibility.

Democratizes access to data science tools.

Abstract

Social science research increasingly demands data-driven insights, yet researchers often face barriers such as lack of technical expertise, inconsistent data formats, and limited access to reliable datasets.Social science research increasingly demands data-driven insights, yet researchers often face barriers such as lack of technical expertise, inconsistent data formats, and limited access to reliable datasets. In this paper, we present a Datalake infrastructure tailored to the needs of interdisciplinary social science research. Our system supports ingestion and integration of diverse data types, automatic provenance and version tracking, role-based access control, and built-in tools for visualization and analysis. We demonstrate the utility of our Datalake using real-world use cases spanning governance, health, and education. A detailed walkthrough of one such use case -- analyzing the relationship between income, education, and infant mortality -- shows how our platform streamlines the research process while maintaining transparency and reproducibility. We argue that such infrastructure can democratize access to advanced data science practices, especially for NGOs, students, and grassroots organizations. The Datalake continues to evolve with plans to support ML pipelines, mobile access, and citizen data feedback mechanisms.