NepaliGPT: A Generative Language Model for the Nepali Language

TL;DR

This paper introduces NepaliGPT, the first large language model for Nepali, built on a new corpus and benchmark dataset, achieving promising results in text generation and coherence.

Contribution

NepaliGPT is the first Nepali-specific generative language model, utilizing a new corpus and benchmark dataset to advance Nepali NLP research.

Findings

Perplexity of 26.32 in text generation

ROUGE-1 score of 0.2604

Causal coherence of 81.25%

Abstract

After the release of ChatGPT, Large Language Models (LLMs) have gained huge popularity in recent days and thousands of variants of LLMs have been released. However, there is no generative language model for the Nepali language, due to which other downstream tasks, including fine-tuning, have not been explored yet. To fill this research gap in the Nepali NLP space, this research proposes \textit{NepaliGPT}, a generative large language model tailored specifically for the Nepali language. This research introduces an advanced corpus for the Nepali language collected from several sources, called the Devanagari Corpus. Likewise, the research introduces the first NepaliGPT benchmark dataset comprised of 4,296 question-answer pairs in the Nepali language. The proposed LLM NepaliGPT achieves the following metrics in text generation: Perplexity of 26.32245, ROUGE-1 score of 0.2604, causal coherence of 81.25\%, and causal consistency of 85.41\%.