Grandes Modelos de Linguagem Multimodais (MLLMs): Da Teoria \`a Pr\'atica

TL;DR

This chapter provides an overview of Multimodal Large Language Models, covering fundamental concepts, key models, practical techniques, and future challenges in integrating language with perception modalities like images and audio.

Contribution

It offers a comprehensive synthesis of MLLMs, including theoretical foundations, emblematic models, and practical methods for implementation and pipeline construction.

Findings

Summarizes key MLLM models and their capabilities.

Provides practical techniques for preprocessing and prompt engineering.

Discusses challenges and promising future trends in MLLMs.

Abstract

Multimodal Large Language Models (MLLMs) combine the natural language understanding and generation capabilities of LLMs with perception skills in modalities such as image and audio, representing a key advancement in contemporary AI. This chapter presents the main fundamentals of MLLMs and emblematic models. Practical techniques for preprocessing, prompt engineering, and building multimodal pipelines with LangChain and LangGraph are also explored. For further practical study, supplementary material is publicly available online: https://github.com/neemiasbsilva/MLLMs-Teoria-e-Pratica. Finally, the chapter discusses the challenges and highlights promising trends.