Efficient Environmental Claim Detection with Hyperbolic Graph Neural Networks

TL;DR

This paper introduces a graph neural network approach, especially hyperbolic GNNs, for environmental claim detection that outperforms transformer models while using significantly fewer parameters, making it suitable for resource-constrained settings.

Contribution

The work presents a novel graph-based method using hyperbolic GNNs for environmental claim detection, demonstrating superior performance and efficiency over existing transformer-based models.

Findings

HGNNs outperform state-of-the-art models in accuracy.

P-HGNNs use up to 30x fewer parameters.

Hierarchical modeling significantly improves HGNN performance.

Abstract

Transformer based models, especially large language models (LLMs) dominate the field of NLP with their mass adoption in tasks such as text generation, summarization and fake news detection. These models offer ease of deployment and reliability for most applications, however, they require significant amounts of computational power for training as well as inference. This poses challenges in their adoption in resource-constrained applications, especially in the open-source community where compute availability is usually scarce. This work proposes a graph-based approach for Environmental Claim Detection, exploring Graph Neural Networks (GNNs) and Hyperbolic Graph Neural Networks (HGNNs) as lightweight yet effective alternatives to transformer-based models. Re-framing the task as a graph classification problem, we transform claim sentences into dependency parsing graphs, utilizing a combination of word2vec \& learnable part-of-speech (POS) tag embeddings for the node features and encoding syntactic dependencies in the edge relations. Our results show that our graph-based models, particularly HGNNs in the poincar\'e space (P-HGNNs), achieve performance superior to the state-of-the-art on environmental claim detection while using up to \textbf{30x fewer parameters}. We also demonstrate that HGNNs benefit vastly from explicitly modeling data in hierarchical (tree-like) structures, enabling them to significantly improve over their euclidean counterparts.