Domain Specific Benchmarks for Evaluating Multimodal Large Language Models

TL;DR

This paper presents a taxonomy of seven key disciplines, reviews domain-specific LLM benchmarks, and categorizes them to facilitate targeted evaluation and advancement of multimodal large language models across various fields.

Contribution

It introduces a domain-specific taxonomy and compiles benchmarks, addressing the gap in domain-focused evaluation of multimodal large language models.

Findings

Identified seven key disciplines for LLM application.

Provided a comprehensive review of domain-specific benchmarks.

Created a categorized resource for future research.

Abstract

Large language models (LLMs) are increasingly being deployed across disciplines due to their advanced reasoning and problem solving capabilities. To measure their effectiveness, various benchmarks have been developed that measure aspects of LLM reasoning, comprehension, and problem-solving. While several surveys address LLM evaluation and benchmarks, a domain-specific analysis remains underexplored in the literature. This paper introduces a taxonomy of seven key disciplines, encompassing various domains and application areas where LLMs are extensively utilized. Additionally, we provide a comprehensive review of LLM benchmarks and survey papers within each domain, highlighting the unique capabilities of LLMs and the challenges faced in their application. Finally, we compile and categorize these benchmarks by domain to create an accessible resource for researchers, aiming to pave the way for advancements toward artificial general intelligence (AGI)