Social Media and Artificial Intelligence for Sustainable Cities and Societies: A Water Quality Analysis Use-case

TL;DR

This paper introduces an NLP framework utilizing social media data to enhance water quality analysis, addressing limitations of traditional surveys through text classification and topic modeling.

Contribution

It presents a novel NLP-based approach combining LLMs and BERTopic for automatic water-related social media analysis, with a large annotated dataset for future research.

Findings

Effective classification of water-related social media posts.

Discovery of regional water concerns through topic modeling.

Large-scale annotated dataset for water quality social media analysis.

Abstract

This paper focuses on a very important societal challenge of water quality analysis. Being one of the key factors in the economic and social development of society, the provision of water and ensuring its quality has always remained one of the top priorities of public authorities. To ensure the quality of water, different methods for monitoring and assessing the water networks, such as offline and online surveys, are used. However, these surveys have several limitations, such as the limited number of participants and low frequency due to the labor involved in conducting such surveys. In this paper, we propose a Natural Language Processing (NLP) framework to automatically collect and analyze water-related posts from social media for data-driven decisions. The proposed framework is composed of two components, namely (i) text classification, and (ii) topic modeling. For text classification, we propose a merit-fusion-based framework incorporating several Large Language Models (LLMs) where different weight selection and optimization methods are employed to assign weights to the LLMs. In topic modeling, we employed the BERTopic library to discover the hidden topic patterns in the water-related tweets. We also analyzed relevant tweets originating from different regions and countries to explore global, regional, and country-specific issues and water-related concerns. We also collected and manually annotated a large-scale dataset, which is expected to facilitate future research on the topic.