UnibucKernel: Geolocating Swiss German Jodels Using Ensemble Learning

TL;DR

This paper presents UnibucKernel, an ensemble learning approach for geolocating Swiss German Jodels, achieving competitive median distance errors by combining diverse models and features.

Contribution

We introduce a novel ensemble method using XGBoost to combine multiple regression models and features for improved geolocation accuracy of Swiss German social media posts.

Findings

Achieved median distance of 23.6 km on test data

Placed third in the 2021 VarDial geolocation task

Ensemble outperforms individual models in accuracy

Abstract

In this work, we describe our approach addressing the Social Media Variety Geolocation task featured in the 2021 VarDial Evaluation Campaign. We focus on the second subtask, which is based on a data set formed of approximately 30 thousand Swiss German Jodels. The dialect identification task is about accurately predicting the latitude and longitude of test samples. We frame the task as a double regression problem, employing an XGBoost meta-learner with the combined power of a variety of machine learning approaches to predict both latitude and longitude. The models included in our ensemble range from simple regression techniques, such as Support Vector Regression, to deep neural models, such as a hybrid neural network and a neural transformer. To minimize the prediction error, we approach the problem from a few different perspectives and consider various types of features, from low-level character n-grams to high-level BERT embeddings. The XGBoost ensemble resulted from combining the power of the aforementioned methods achieves a median distance of 23.6 km on the test data, which places us on the third place in the ranking, at a difference of 6.05 km and 2.9 km from the submissions on the first and second places, respectively.