Smartflow: Enabling Scalable Spatiotemporal Geospatial Research

TL;DR

Smartflow is a scalable, cloud-based framework that standardizes and streamlines spatiotemporal geospatial data processing, model experimentation, and analysis, facilitating large-scale geospatial research and monitoring.

Contribution

It introduces Smartflow, a novel, open-source, Kubernetes-based framework for scalable geospatial data processing and model development, including a new neural architecture for construction monitoring.

Findings

Smartflow enables processing of large geospatial datasets efficiently.

The neural model successfully detects heavy construction phases.

Framework supports extensive geographic and temporal scales.

Abstract

BlackSky introduces Smartflow, a cloud-based framework enabling scalable spatiotemporal geospatial research built on open-source tools and technologies. Using STAC-compliant catalogs as a common input, heterogeneous geospatial data can be processed into standardized datacubes for analysis and model training. Model experimentation is managed using a combination of tools, including ClearML, Tensorboard, and Apache Superset. Underpinning Smartflow is Kubernetes, which orchestrates the provisioning and execution of workflows to support both horizontal and vertical scalability. This combination of features makes Smartflow well-suited for geospatial model development and analysis over large geographic areas, time scales, and expansive image archives. We also present a novel neural architecture, built using Smartflow, to monitor large geographic areas for heavy construction. Qualitative results based on data from the IARPA Space-based Machine Automated Recognition Technique (SMART) program are presented that show the model is capable of detecting heavy construction throughout all major phases of development.