Advancing Generative AI for Portuguese with Open Decoder Gerv\'asio PT*

TL;DR

This paper introduces Gervásio PT*, an open-source, instruction-tuned Transformer model that achieves state-of-the-art neural decoding for Portuguese, built on LLaMA 2 and enhanced with new Portuguese instruction datasets.

Contribution

It presents a fully open, high-performing Portuguese decoder model based on LLaMA 2, with new instruction datasets and open licensing for research and commercial use.

Findings

Gervásio PT* sets a new state of the art in Portuguese neural decoding.

The model is open source and accessible for research and commercial purposes.

It can be run on consumer-grade hardware.

Abstract

To advance the neural decoding of Portuguese, in this paper we present a fully open Transformer-based, instruction-tuned decoder model that sets a new state of the art in this respect. To develop this decoder, which we named Gerv\'asio PT*, a strong LLaMA~2 7B model was used as a starting point, and its further improvement through additional training was done over language resources that include new instruction data sets of Portuguese prepared for this purpose, which are also contributed in this paper. All versions of Gerv\'asio are open source and distributed for free under an open license, including for either research or commercial usage, and can be run on consumer-grade hardware, thus seeking to contribute to the advancement of research and innovation in language technology for Portuguese.