Big Data: How Geo-information Helped Shape the Future of Data Engineering

TL;DR

This paper reviews the evolution of geo-information sciences, highlighting how large data sets from remote sensing and GIS have influenced data engineering, addressing challenges like data structure, processing, uncertainty, consistency, and ontologies.

Contribution

It provides a retrospective analysis of key issues faced in geo-information sciences and their impact on the development of data engineering and internet query technologies.

Findings

Large datasets have driven advancements in data processing and storage.

Addressing uncertainty and consistency has been crucial for reliable geo-information.

Ontologies have played a key role in standardizing definitions and decision-making.

Abstract

Very large data sets are the common rule in automated mapping, GIS, remote sensing, and what we can name geo-information. Indeed, in 1983 Landsat was already delivering gigabytes of data, and other sensors were in orbit or ready for launch, and a tantamount of cartographic data was being digitized. The retrospective paper revisits several issues that geo-information sciences had to face from the early stages on, including: structure ( to bring some structure to the data registered from a sampled signal, metadata); processing (huge amounts of data for big computers and fast algorithms); uncertainty (the kinds of errors, their quantification); consistency (when merging different sources of data is logically allowed, and meaningful); ontologies (clear and agreed shared definitions, if any kind of decision should be based upon them). All these issues are the background of Internet queries, and the underlying technology has been shaped during those years when geo-information engineering emerged.