Data Transformation Strategies to Remove Heterogeneity

TL;DR

This paper reviews data transformation strategies to address heterogeneity caused by data format differences, emphasizing their importance in AI data preparation and highlighting challenges in selecting suitable techniques.

Contribution

It systematically categorizes and analyzes contemporary data transformation approaches for handling data heterogeneity in AI applications.

Findings

Identifies key sources of data heterogeneity.

Classifies transformation strategies based on their approaches.

Highlights challenges and considerations in strategy selection.

Abstract

Data heterogeneity is a prevalent issue, stemming from various conflicting factors, making its utilization complex. This uncertainty, particularly resulting from disparities in data formats, frequently necessitates the involvement of experts to find resolutions. Current methodologies primarily address conflicts related to data structures and schemas, often overlooking the pivotal role played by data transformation. As the utilization of artificial intelligence (AI) continues to expand, there is a growing demand for a more streamlined data preparation process, and data transformation becomes paramount. It customizes training data to enhance AI learning efficiency and adapts input formats to suit diverse AI models. Selecting an appropriate transformation technique is paramount in preserving crucial data details. Despite the widespread integration of AI across various industries, comprehensive reviews concerning contemporary data transformation approaches are scarce. This survey explores the intricacies of data heterogeneity and its underlying sources. It systematically categorizes and presents strategies to address heterogeneity stemming from differences in data formats, shedding light on the inherent challenges associated with each strategy.