Multidimensional classification of posts for online course discussion forum curation

TL;DR

This paper introduces a Bayesian fusion method to improve online course discussion forum classification by combining generic LLM scores with local data classifiers, reducing the need for costly fine-tuning.

Contribution

It presents a novel fusion approach that enhances classification performance without requiring frequent LLM retraining, offering a resource-efficient alternative.

Findings

Fusion outperforms individual classifiers

Comparable to fine-tuned LLMs in accuracy

Reduces retraining costs for online forum curation

Abstract

The automatic curation of discussion forums in online courses requires constant updates, making frequent retraining of Large Language Models (LLMs) a resource-intensive process. To circumvent the need for costly fine-tuning, this paper proposes and evaluates the use of Bayesian fusion. The approach combines the multidimensional classification scores of a pre-trained generic LLM with those of a classifier trained on local data. The performance comparison demonstrated that the proposed fusion improves the results compared to each classifier individually, and is competitive with the LLM fine-tuning approach