Open Sentence Embeddings for Portuguese with the Serafim PT* encoders family

TL;DR

This paper introduces Serafim PT*, a family of open-source Portuguese sentence encoders that achieve state-of-the-art performance and are adaptable to different hardware, with a systematic study on their design choices.

Contribution

It presents a new family of Portuguese sentence encoders with diverse sizes and a comprehensive analysis of their training objectives and parameters.

Findings

State-of-the-art performance across models

Open-source availability for research and commercial use

Insights on learning objectives and parameter selection

Abstract

Sentence encoder encode the semantics of their input, enabling key downstream applications such as classification, clustering, or retrieval. In this paper, we present Serafim PT*, a family of open-source sentence encoders for Portuguese with various sizes, suited to different hardware/compute budgets. Each model exhibits state-of-the-art performance and is made openly available under a permissive license, allowing its use for both commercial and research purposes. Besides the sentence encoders, this paper contributes a systematic study and lessons learned concerning the selection criteria of learning objectives and parameters that support top-performing encoders.