Automated river gauge plate reading using a hybrid object detection and generative AI framework in the Limpopo River Basin

TL;DR

This paper introduces a hybrid AI framework combining vision detection and large language models to automate river gauge readings, significantly improving accuracy and efficiency in water level monitoring.

Contribution

It presents a novel integrated approach using vision detection and multimodal LLMs for automated, accurate river gauge reading, enhancing traditional hydrological monitoring methods.

Findings

Waterline detection achieved 94.24% precision and 83.64% F1 score.

Gemini Stage 2 LLM had a mean absolute error of 5.43 cm.

Combining geometric metadata with LLMs improves water level estimation accuracy.

Abstract

Accurate and continuous monitoring of river water levels is essential for flood forecasting, water resource management, and ecological protection. Traditional hydrological observation methods are often limited by manual measurement errors and environmental constraints. This study presents a hybrid framework integrating vision based waterline detection, YOLOv8 pose scale extraction, and large multimodal language models (GPT 4o and Gemini 2.0 Flash) for automated river gauge plate reading. The methodology involves sequential stages of image preprocessing, annotation, waterline detection, scale gap estimation, and numeric reading extraction. Experiments demonstrate that waterline detection achieved high precision of 94.24 percent and an F1 score of 83.64 percent, while scale gap detection provided accurate geometric calibration for subsequent reading extraction. Incorporating scale gap metadata substantially improved the predictive performance of LLMs, with Gemini Stage 2 achieving the highest accuracy, with a mean absolute error of 5.43 cm, root mean square error of 8.58 cm, and R squared of 0.84 under optimal image conditions. Results highlight the sensitivity of LLMs to image quality, with degraded images producing higher errors, and underscore the importance of combining geometric metadata with multimodal artificial intelligence for robust water level estimation. Overall, the proposed approach offers a scalable, efficient, and reliable solution for automated hydrological monitoring, demonstrating potential for real time river gauge digitization and improved water resource management.