Isotropy-Optimized Contrastive Learning for Semantic Course Recommendation

TL;DR

This paper introduces an isotropy-regularized contrastive learning method based on BERT to generate more discriminative course embeddings, significantly improving semantic course recommendations by addressing anisotropy in traditional BERT representations.

Contribution

It proposes a novel contrastive learning framework with data augmentation and isotropy regularization to enhance BERT embeddings for semantic course recommendation.

Findings

Improved embedding separation over vanilla BERT

More accurate course recommendations achieved

Enhanced discriminative power of course embeddings

Abstract

This paper presents a semantic course recommendation system for students using a self-supervised contrastive learning approach built upon BERT (Bidirectional Encoder Representations from Transformers). Traditional BERT embeddings suffer from anisotropic representation spaces, where course descriptions exhibit high cosine similarities regardless of semantic relevance. To address this limitation, we propose a contrastive learning framework with data augmentation and isotropy regularization that produces more discriminative embeddings. Our system processes student text queries and recommends Top-N relevant courses from a curated dataset of over 500 engineering courses across multiple faculties. Experimental results demonstrate that our fine-tuned model achieves improved embedding separation and more accurate course recommendations compared to vanilla BERT baselines.