Compiling and Processing Historical and Contemporary Portuguese Corpora

TL;DR

This paper details the framework for processing large Portuguese corpora, including pre-processing, annotation, and querying methods, covering both contemporary and historical texts from Brazil, Portugal, and the 16th-20th centuries.

Contribution

It introduces a comprehensive framework for processing and analyzing diverse Portuguese corpora, including historical texts, with detailed methods and applications.

Findings

Effective pre-processing and annotation techniques for Portuguese corpora

Development of indexing and querying methods for large texts

Use of corpora in published research papers

Abstract

This technical report describes the framework used for processing three large Portuguese corpora. Two corpora contain texts from newspapers, one published in Brazil and the other published in Portugal. The third corpus is Colonia, a historical Portuguese collection containing texts written between the 16th and the early 20th century. The report presents pre-processing methods, segmentation, and annotation of the corpora as well as indexing and querying methods. Finally, it presents published research papers using the corpora.