MASS-UMAP: Fast and accurate analog ensemble search in weather radar archive

TL;DR

This paper introduces MASS-UMAP, a novel method combining UMAP and MASS algorithms to enable rapid and accurate analog pattern retrieval in weather radar archives, significantly improving operational efficiency.

Contribution

The paper presents a new architecture that combines UMAP and MASS for fast, accurate spatiotemporal analog search in radar data, outperforming traditional methods.

Findings

UMAP outperforms PCA in analog search accuracy.

MASS is 20 times faster than brute force search.

The system retrieves analogs in less than 5 seconds for 2 years of data.

Abstract

The use of analogs - similar weather patterns - for weather forecasting and analysis is an established method in meteorology. The most challenging aspect of using this approach in the context of operational radar applications is to be able to perform a fast and accurate search for similar spatiotemporal precipitation patterns in a large archive of historical records. In this context, sequential pairwise search is too slow and computationally expensive. Here we propose an architecture to significantly speed-up spatiotemporal analog retrieval by combining nonlinear geometric dimensionality reduction (UMAP) with the fastest known Euclidean search algorithm for time series (MASS) to find radar analogs in constant time, independently of the desired temporal length to match and the number of extracted analogs. We compare UMAP with Principal component analysis (PCA) and show that UMAP outperforms PCA for spatial MSE analog search with proper settings. Moreover, we show that MASS is 20 times faster than brute force search on the UMAP embeddings space. We test the architecture on a real dataset and show that it enables precise and fast operational analog ensemble search through more than 2 years of radar archive in less than 5 seconds on a single workstation.