Understanding Student Sentiment on Mental Health Support in Colleges Using Large Language Models

TL;DR

This paper leverages large language models to analyze student sentiments on college mental health support, creating a new dataset and demonstrating the effectiveness of GPT-3.5 and BERT for sentiment analysis in this context.

Contribution

It introduces SMILE-College, a sentiment analysis dataset created through human-machine collaboration, and evaluates LLMs for mental health support assessment.

Findings

GPT-3.5 and BERT outperform traditional methods

LLMs can improve mental health support evaluation

Challenges remain in sentiment prediction accuracy

Abstract

Mental health support in colleges is vital in educating students by offering counseling services and organizing supportive events. However, evaluating its effectiveness faces challenges like data collection difficulties and lack of standardized metrics, limiting research scope. Student feedback is crucial for evaluation but often relies on qualitative analysis without systematic investigation using advanced machine learning methods. This paper uses public Student Voice Survey data to analyze student sentiments on mental health support with large language models (LLMs). We created a sentiment analysis dataset, SMILE-College, with human-machine collaboration. The investigation of both traditional machine learning methods and state-of-the-art LLMs showed the best performance of GPT-3.5 and BERT on this new dataset. The analysis highlights challenges in accurately predicting response sentiments and offers practical insights on how LLMs can enhance mental health-related research and improve college mental health services. This data-driven approach will facilitate efficient and informed mental health support evaluation, management, and decision-making.