FiLLM -- A Filipino-optimized Large Language Model based on Southeast Asia Large Language Model (SEALLM)

TL;DR

FiLLM is a Filipino-optimized large language model built on SeaLLM-7B, utilizing LoRA fine-tuning to improve NLP tasks in Filipino, with evaluations showing its effectiveness and scalability for local language processing.

Contribution

The paper introduces FiLLM, a novel Filipino-optimized LLM based on SeaLLM-7B, employing LoRA fine-tuning for efficient NLP in Filipino language tasks.

Findings

FiLLM performs effectively on Filipino NLP tasks.

Compared to CalamanCy, FiLLM shows competitive performance.

The model demonstrates scalability and efficiency for local language applications.

Abstract

This study presents FiLLM, a Filipino-optimized large language model, designed to enhance natural language processing (NLP) capabilities in the Filipino language. Built upon the SeaLLM-7B 2.5 model, FiLLM leverages Low-Rank Adaptation (LoRA) fine-tuning to optimize memory efficiency while maintaining task-specific performance. The model was trained and evaluated on diverse Filipino datasets to address key NLP tasks, including Named Entity Recognition (NER), Part-of-Speech (POS) tagging, Dependency Parsing, and Text Summarization. Performance comparisons with the CalamanCy model were conducted using F1 Score, Precision, Recall, Compression Rate, and Keyword Overlap metrics. Results indicate that Calamancy outperforms FILLM in several aspects, demonstrating its effectiveness in processing Filipino text with improved linguistic comprehension and adaptability. This research contributes to the advancement of Filipino NLP applications by providing an optimized, efficient, and scalable language model tailored for local linguistic needs.