METIS: Mentoring Engine for Thoughtful Inquiry & Solutions

TL;DR

METIS is an AI-powered mentoring tool designed to guide undergraduates from initial ideas to research papers through stage-aware assistance, outperforming existing large language models in various evaluation metrics.

Contribution

This work introduces METIS, a novel stage-aware AI mentoring system with integrated literature search and methodology checks, improving research guidance for students.

Findings

METIS outperforms Claude Sonnet 4.5 and GPT-5 in preference tests.

Student scores on clarity and actionability are higher with METIS.

METIS achieves slightly higher final quality in multi-turn tutoring scenarios.

Abstract

Many students lack access to expert research mentorship. We ask whether an AI mentor can move undergraduates from an idea to a paper. We build METIS, a tool-augmented, stage-aware assistant with literature search, curated guidelines, methodology checks, and memory. We evaluate METIS against GPT-5 and Claude Sonnet 4.5 across six writing stages using LLM-as-a-judge pairwise preferences, student-persona rubrics, short multi-turn tutoring, and evidence/compliance checks. On 90 single-turn prompts, LLM judges preferred METIS to Claude Sonnet 4.5 in 71% and to GPT-5 in 54%. Student scores (clarity/actionability/constraint-fit; 90 prompts x 3 judges) are higher across stages. In multi-turn sessions (five scenarios/agent), METIS yields slightly higher final quality than GPT-5. Gains concentrate in document-grounded stages (D-F), consistent with stage-aware routing and groundings failure modes include premature tool routing, shallow grounding, and occasional stage misclassification.