Harnessing GPT-4V(ision) for Insurance: A Preliminary Exploration

TL;DR

This paper investigates GPT-4V's capabilities in handling multimodal insurance tasks, highlighting its strengths in understanding insurance scenarios and identifying limitations like hallucinations and language support issues.

Contribution

It provides a systematic exploration of GPT-4V's performance across diverse insurance-related multimodal tasks, a novel application in this domain.

Findings

GPT-4V demonstrates strong understanding of insurance scenarios.

It struggles with detailed risk rating and loss assessment.

The model exhibits hallucinations in image understanding.

Abstract

The emergence of Large Multimodal Models (LMMs) marks a significant milestone in the development of artificial intelligence. Insurance, as a vast and complex discipline, involves a wide variety of data forms in its operational processes, including text, images, and videos, thereby giving rise to diverse multimodal tasks. Despite this, there has been limited systematic exploration of multimodal tasks specific to insurance, nor a thorough investigation into how LMMs can address these challenges. In this paper, we explore GPT-4V's capabilities in the insurance domain. We categorize multimodal tasks by focusing primarily on visual aspects based on types of insurance (e.g., auto, household/commercial property, health, and agricultural insurance) and insurance stages (e.g., risk assessment, risk monitoring, and claims processing). Our experiment reveals that GPT-4V exhibits remarkable abilities in insurance-related tasks, demonstrating not only a robust understanding of multimodal content in the insurance domain but also a comprehensive knowledge of insurance scenarios. However, there are notable shortcomings: GPT-4V struggles with detailed risk rating and loss assessment, suffers from hallucination in image understanding, and shows variable support for different languages. Through this work, we aim to bridge the insurance domain with cutting-edge LMM technology, facilitate interdisciplinary exchange and development, and provide a foundation for the continued advancement and evolution of future research endeavors.