AviationLMM: A Large Multimodal Foundation Model for Civil Aviation

TL;DR

AviationLMM is a large multimodal foundation model designed to unify heterogeneous civil aviation data streams, enhancing situational awareness, decision support, and safety through integrated understanding and reasoning.

Contribution

The paper introduces the architecture and design of AviationLMM, a multimodal foundation model that processes diverse aviation data for improved AI applications in civil aviation.

Findings

Model architecture supports cross-modal alignment and fusion.

Enables generation of summaries, alerts, diagnostics, and incident reconstructions.

Identifies key research challenges in data, reasoning, trustworthiness, and privacy.

Abstract

Civil aviation is a cornerstone of global transportation and commerce, and ensuring its safety, efficiency and customer satisfaction is paramount. Yet conventional Artificial Intelligence (AI) solutions in aviation remain siloed and narrow, focusing on isolated tasks or single modalities. They struggle to integrate heterogeneous data such as voice communications, radar tracks, sensor streams and textual reports, which limits situational awareness, adaptability, and real-time decision support. This paper introduces the vision of AviationLMM, a Large Multimodal foundation Model for civil aviation, designed to unify the heterogeneous data streams of civil aviation and enable understanding, reasoning, generation and agentic applications. We firstly identify the gaps between existing AI solutions and requirements. Secondly, we describe the model architecture that ingests multimodal inputs such as air-ground voice, surveillance, on-board telemetry, video and structured texts, and performs cross-modal alignment and fusion, and produces flexible outputs ranging from situation summaries and risk alerts to predictive diagnostics and multimodal incident reconstructions. In order to fully realize this vision, we identify key research opportunities to address, including data acquisition, alignment and fusion, pretraining, reasoning, trustworthiness, privacy, robustness to missing modalities, and synthetic scenario generation. By articulating the design and challenges of AviationLMM, we aim to boost the civil aviation foundation model progress and catalyze coordinated research efforts toward an integrated, trustworthy and privacy-preserving aviation AI ecosystem.