Quantifying the Effectiveness of Student Organization Activities using Natural Language Processing

TL;DR

This paper develops a sentiment analysis workflow using BERT to quantify the effectiveness of student activities based on emotional responses, demonstrating NLP's practical application in educational settings.

Contribution

It introduces a novel machine learning workflow utilizing BERT for analyzing student emotional responses to measure activity effectiveness.

Findings

BERT-based sentiment analysis effectively quantifies student activity impact.

The workflow produces an Event Score reflecting activity success.

NLP can be applied beyond commercial contexts to educational assessment.

Abstract

Student extracurricular activities play an important role in enriching the students' educational experiences. With the increasing popularity of Machine Learning and Natural Language Processing, it becomes a logical step that incorporating ML-NLP in improving extracurricular activities is a potential focus of study in Artificial Intelligence (AI). This research study aims to develop a machine learning workflow that will quantify the effectiveness of student-organized activities based on student emotional responses using sentiment analysis. The study uses the Bidirectional Encoder Representations from Transformers (BERT) Large Language Model (LLM) called via the pysentimiento toolkit, as a Transformer pipeline in Hugging Face. A sample data set from Organization C, a Recognized Student Organization (RSO) of a higher educational institute in the Philippines, College X, was used to develop the workflow. The workflow consisted of data preprocessing, key feature selection, LLM feature processing, and score aggregation, resulting in an Event Score for each data set. The results show that the BERT LLM can also be used effectively in analyzing sentiment beyond product reviews and post comments. For the student affairs offices of educational institutions, this study can provide a practical example of how NLP can be applied to real-world scenarios, showcasing the potential impact of data-driven decision making.