Transformadores: Fundamentos teoricos y Aplicaciones

TL;DR

This paper provides a comprehensive overview of transformer models, detailing their theoretical foundations, core components, modifications, and applications across various data types, aimed at Spanish-speaking researchers.

Contribution

It offers an in-depth explanation of transformer architecture, including mathematical and algorithmic foundations, and discusses recent research and applications in a Spanish-language resource.

Findings

Explains the self-attention mechanism and its evolution.

Details various architectural modifications and their implications.

Highlights diverse applications across data modalities.

Abstract

Transformers are a neural network architecture originally developed for natural language processing, which have since become a foundational tool for solving a wide range of problems, including text, audio, image processing, reinforcement learning, and other tasks involving heterogeneous input data. Their hallmark is the self-attention mechanism, which allows the model to weigh different parts of the input sequence dynamically, and is an evolution of earlier attention-based approaches. This article provides readers with the necessary background to understand recent research on transformer models, and presents the mathematical and algorithmic foundations of their core components. It also explores the architecture's various elements, potential modifications, and some of the most relevant applications. The article is written in Spanish to help make this scientific knowledge more accessible to the Spanish-speaking community.