A generic framework for the development of geospatial processing pipelines on clusters

TL;DR

This paper introduces a flexible, open-source framework that simplifies the development of geospatial processing pipelines on clusters, enabling efficient and scalable Earth Observation data analysis.

Contribution

It presents a generic, open-source framework that facilitates parallelization of geospatial processing pipelines on clusters, reducing development effort and improving performance.

Findings

Framework enables transparent parallel processing of RS images

Demonstrates significant performance improvements on typical RS tasks

Supports scalable processing of large Earth Observation datasets

Abstract

The amount of remote sensing data available to applications is constantly growing due to the rise of very-high-resolution sensors and short repeat cycle satellites. Consequently, tackling computational complexity in Earth Observation information extraction is rising as a major challenge. Resorting to High Performance Computing (HPC) is becoming a common practice, since it provides environments and programming facilities able to speed-up processes. In particular, clusters are flexible, cost-effective systems able to perform data-intensive tasks ideally fulfilling any computational requirement. However, their use typically implies a significant coding effort to build proper implementations of specific processing pipelines. This paper presents a generic framework for the development of RS images processing applications targeting cluster computing. It is based on common open sources libraries, and leverages the parallelization of a wide variety of image processing pipelines in a transparent way. Performances on typical RS tasks implemented using the proposed framework demonstrate a great potential for the effective and timely processing of large amount of data.