From Struggle (06-2024) to Mastery (02-2025) LLMs Conquer Advanced Algorithm Exams and Pave the Way for Editorial Generation

TL;DR

This study evaluates state-of-the-art LLMs on university-level algorithm exams, showing they can perform at top student levels and support educational content creation, despite some challenges with graph tasks.

Contribution

It provides a comprehensive empirical analysis of LLMs' problem-solving abilities on complex algorithm exams and explores their potential in educational content generation.

Findings

LLMs achieve top student-level scores on algorithm exams

Models demonstrate strong reasoning on multi-step problems

Difficulties remain with graph-based tasks

Abstract

This paper presents a comprehensive evaluation of the performance of state-of-the-art Large Language Models (LLMs) on challenging university-level algorithms exams. By testing multiple models on both a Romanian exam and its high-quality English translation, we analyze LLMs' problem-solving capabilities, consistency, and multilingual performance. Our empirical study reveals that the most recent models not only achieve scores comparable to top-performing students but also demonstrate robust reasoning skills on complex, multi-step algorithmic challenges, even though difficulties remain with graph-based tasks. Building on these findings, we explore the potential of LLMs to support educational environments through the generation of high-quality editorial content, offering instructors a powerful tool to enhance student feedback. The insights and best practices discussed herein pave the way for further integration of generative AI in advanced algorithm education.