Unipa-GPT: Large Language Models for university-oriented QA in Italian

TL;DR

Unipa-GPT is a university-specific Italian language chatbot built on GPT-3.5-turbo, designed to assist students with course selection through retrieval-augmented generation and fine-tuning, evaluated during a public event.

Contribution

This work introduces Unipa-GPT, a novel university-oriented Italian LLM chatbot utilizing RAG and fine-tuning, with comparative analysis and public deployment insights.

Findings

RAG and fine-tuning approaches both effective

Unipa-GPT performs well in university-specific queries

Open-source code and data available for further research

Abstract

This paper illustrates the architecture and training of Unipa-GPT, a chatbot relying on a Large Language Model, developed for assisting students in choosing a bachelor/master degree course at the University of Palermo. Unipa-GPT relies on gpt-3.5-turbo, it was presented in the context of the European Researchers' Night (SHARPER night). In our experiments we adopted both the Retrieval Augmented Generation (RAG) approach and fine-tuning to develop the system. The whole architecture of Unipa-GPT is presented, both the RAG and the fine-tuned systems are compared, and a brief discussion on their performance is reported. Further comparison with other Large Language Models and the experimental results during the SHARPER night are illustrated. Corpora and code are available on GitHub