AgriBench: A Hierarchical Agriculture Benchmark for Multimodal Large Language Models

TL;DR

This paper introduces AgriBench, a comprehensive benchmark for evaluating multimodal large language models in agriculture, supported by a new detailed dataset called MM-LUCAS that includes images, annotations, and land use data.

Contribution

The paper presents the first agriculture-specific benchmark for multimodal LLMs and introduces MM-LUCAS, a rich dataset with diverse agricultural and geographical annotations.

Findings

AgriBench enables systematic evaluation of agriculture MM-LLMs.

MM-LUCAS provides extensive multimodal agricultural data.

The work offers insights for future agriculture AI developments.

Abstract

We introduce AgriBench, the first agriculture benchmark designed to evaluate MultiModal Large Language Models (MM-LLMs) for agriculture applications. To further address the agriculture knowledge-based dataset limitation problem, we propose MM-LUCAS, a multimodal agriculture dataset, that includes 1,784 landscape images, segmentation masks, depth maps, and detailed annotations (geographical location, country, date, land cover and land use taxonomic details, quality scores, aesthetic scores, etc), based on the Land Use/Cover Area Frame Survey (LUCAS) dataset, which contains comparable statistics on land use and land cover for the European Union (EU) territory. This work presents a groundbreaking perspective in advancing agriculture MM-LLMs and is still in progress, offering valuable insights for future developments and innovations in specific expert knowledge-based MM-LLMs.