WebUIBench: A Comprehensive Benchmark for Evaluating Multimodal Large Language Models in WebUI-to-Code

TL;DR

WebUIBench is a comprehensive benchmark designed to evaluate multimodal large language models across multiple web development sub-tasks, providing detailed insights into their strengths and weaknesses in WebUI-to-Code tasks.

Contribution

This work introduces WebUIBench, a novel multi-view evaluation framework with 21K QA pairs, to systematically assess MLLMs' capabilities in web development tasks.

Findings

29 MLLMs evaluated revealing diverse skill profiles.

Identified common weaknesses in perception and understanding.

Benchmark guides future model improvements.

Abstract

With the rapid advancement of Generative AI technology, Multimodal Large Language Models(MLLMs) have the potential to act as AI software engineers capable of executing complex web application development. Considering that the model requires a confluence of multidimensional sub-capabilities to address the challenges of various development phases, constructing a multi-view evaluation framework is crucial for accurately guiding the enhancement of development efficiency. However, existing benchmarks usually fail to provide an assessment of sub-capabilities and focus solely on webpage generation outcomes. In this work, we draw inspiration from the principles of software engineering and further propose WebUIBench, a benchmark systematically designed to evaluate MLLMs in four key areas: WebUI Perception, HTML Programming,WebUI-HTML Understanding, and WebUI-to-Code. WebUIBench comprises 21K high-quality question-answer pairs derived from over 0.7K real-world websites. The extensive evaluation of 29 mainstream MLLMs uncovers the skill characteristics and various weakness that models encountered during the development process.