AdvisingWise: Supporting Academic Advising in Higher Education Settings Through a Human-in-the-Loop Multi-Agent Framework

TL;DR

AdvisingWise is a multi-agent system that enhances academic advising by automating tasks with LLMs while ensuring human oversight, leading to more accurate, personalized support and improved advisor perceptions.

Contribution

This paper introduces AdvisingWise, a novel human-in-the-loop multi-agent framework that combines AI automation with human validation for effective academic advising support.

Findings

AdvisingWise produces accurate, personalized responses.

Advisors' perceptions of the system improved after use.

The system effectively integrates authoritative resources and adaptive prompts.

Abstract

Academic advising is critical to student success in higher education, yet high student-to-advisor ratios limit advisors' capacity to provide timely support, particularly during peak periods. Recent advances in Large Language Models (LLMs) present opportunities to enhance the advising process. We present AdvisingWise, a multi-agent system that automates time-consuming tasks, such as information retrieval and response drafting, while preserving human oversight. AdvisingWise leverages authoritative institutional resources and adaptively prompts students about their academic backgrounds to generate reliable, personalized responses. All system responses undergo human advisor validation before delivery to students. We evaluate AdvisingWise through a mixed-methods approach: (1) expert evaluation on responses of 20 sample queries, (2) LLM-as-a-judge evaluation of the information retrieval strategy, and (3) a user study with 8 academic advisors to assess the system's practical utility. Our evaluation shows that AdvisingWise produces accurate, personalized responses. Advisors reported increasingly positive perceptions after using AdvisingWise, as their initial concerns about reliability and personalization diminished. We conclude by discussing the implications of human-AI synergy on the practice of academic advising.