VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Models

TL;DR

The VNHSGE dataset offers a comprehensive benchmark for evaluating large language models across multiple subjects, including text and images, highlighting current strengths and areas for improvement in LLM performance.

Contribution

This paper introduces the VNHSGE dataset, a new extensive Vietnamese high school exam dataset for evaluating LLMs across diverse tasks and subjects, including multimodal data.

Findings

ChatGPT and BingChat perform at human level in literature, English, history, geography, and civics.

LLMs show room for improvement in mathematics, physics, chemistry, and biology.

The dataset provides a broad benchmark for future LLM development.

Abstract

The VNHSGE (VietNamese High School Graduation Examination) dataset, developed exclusively for evaluating large language models (LLMs), is introduced in this article. The dataset, which covers nine subjects, was generated from the Vietnamese National High School Graduation Examination and comparable tests. 300 literary essays have been included, and there are over 19,000 multiple-choice questions on a range of topics. The dataset assesses LLMs in multitasking situations such as question answering, text generation, reading comprehension, visual question answering, and more by including both textual data and accompanying images. Using ChatGPT and BingChat, we evaluated LLMs on the VNHSGE dataset and contrasted their performance with that of Vietnamese students to see how well they performed. The results show that ChatGPT and BingChat both perform at a human level in a number of areas, including literature, English, history, geography, and civics education. They still have space to grow, though, especially in the areas of mathematics, physics, chemistry, and biology. The VNHSGE dataset seeks to provide an adequate benchmark for assessing the abilities of LLMs with its wide-ranging coverage and variety of activities. We intend to promote future developments in the creation of LLMs by making this dataset available to the scientific community, especially in resolving LLMs' limits in disciplines involving mathematics and the natural sciences.