A New Era in Human Factors Engineering: A Survey of the Applications and Prospects of Large Multimodal Models

TL;DR

This paper surveys the emerging role of large multimodal models in human factors engineering, highlighting applications, challenges, and future research directions in integrating AI with human-centered design.

Contribution

It introduces a novel literature review method and systematically explores the applications and future prospects of LMMs in human factors and ergonomics.

Findings

LMMs are increasingly used in accident analysis and human modeling.

LMMs introduce new research paradigms and methodologies.

Future challenges include integrating LMMs effectively into human factors studies.

Abstract

In recent years, the potential applications of Large Multimodal Models (LMMs) in fields such as healthcare, social psychology, and industrial design have attracted wide research attention, providing new directions for human factors research. For instance, LMM-based smart systems have become novel research subjects of human factors studies, and LMM introduces new research paradigms and methodologies to this field. Therefore, this paper aims to explore the applications, challenges, and future prospects of LMM in the domain of human factors and ergonomics through an expert-LMM collaborated literature review. Specifically, a novel literature review method is proposed, and research studies of LMM-based accident analysis, human modelling and intervention design are introduced. Subsequently, the paper discusses future trends of the research paradigm and challenges of human factors and ergonomics studies in the era of LMMs. It is expected that this study can provide a valuable perspective and serve as a reference for integrating human factors with artificial intelligence.