ProtSi: Prototypical Siamese Network with Data Augmentation for Few-Shot Subjective Answer Evaluation

TL;DR

ProtSi is a semi-supervised, few-shot learning model for subjective answer evaluation that combines Siamese and Prototypical Networks with data augmentation, improving accuracy and interpretability in limited data scenarios.

Contribution

The paper introduces ProtSi, a novel semi-supervised architecture that applies few-shot learning to subjective answer evaluation, integrating data augmentation and contrastive learning for better performance.

Findings

Outperforms recent baselines in accuracy and quadratic weighted kappa

Effective in limited annotated data scenarios

Utilizes data augmentation to prevent overfitting

Abstract

Subjective answer evaluation is a time-consuming and tedious task, and the quality of the evaluation is heavily influenced by a variety of subjective personal characteristics. Instead, machine evaluation can effectively assist educators in saving time while also ensuring that evaluations are fair and realistic. However, most existing methods using regular machine learning and natural language processing techniques are generally hampered by a lack of annotated answers and poor model interpretability, making them unsuitable for real-world use. To solve these challenges, we propose ProtSi Network, a unique semi-supervised architecture that for the first time uses few-shot learning to subjective answer evaluation. To evaluate students' answers by similarity prototypes, ProtSi Network simulates the natural process of evaluator scoring answers by combining Siamese Network which consists of BERT and encoder layers with Prototypical Network. We employed an unsupervised diverse paraphrasing model ProtAugment, in order to prevent overfitting for effective few-shot text classification. By integrating contrastive learning, the discriminative text issue can be mitigated. Experiments on the Kaggle Short Scoring Dataset demonstrate that the ProtSi Network outperforms the most recent baseline models in terms of accuracy and quadratic weighted kappa.