GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark
Dongyang Li, Ruixue Ding, Qiang Zhang, Zheng Li, Boli Chen, Pengjun, Xie, Yao Xu, Xin Li, Ning Guo, Fei Huang, Xiaofeng He

TL;DR
GeoGLUE is a comprehensive benchmark designed to evaluate geographic natural language understanding across six tasks, addressing a gap in standardized evaluation for geographic NLP models.
Contribution
It introduces the first unified benchmark for geographic NLP, with diverse tasks and data sources, facilitating consistent evaluation and comparison of models.
Findings
Baseline experiments demonstrate the benchmark's effectiveness.
The dataset covers six diverse geographic NLP tasks.
GeoGLUE highlights the need for specialized geographic NLP models.
Abstract
With a fast developing pace of geographic applications, automatable and intelligent models are essential to be designed to handle the large volume of information. However, few researchers focus on geographic natural language processing, and there has never been a benchmark to build a unified standard. In this work, we propose a GeoGraphic Language Understanding Evaluation benchmark, named GeoGLUE. We collect data from open-released geographic resources and introduce six natural language understanding tasks, including geographic textual similarity on recall, geographic textual similarity on rerank, geographic elements tagging, geographic composition analysis, geographic where what cut, and geographic entity alignment. We also pro vide evaluation experiments and analysis of general baselines, indicating the effectiveness and significance of the GeoGLUE benchmark.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGeographic Information Systems Studies · Natural Language Processing Techniques · Speech and dialogue systems
