Give me a hint: Can LLMs take a hint to solve math problems?
Vansh Agrawal, Pratham Singla, Amitoj Singh Miglani, Shivank Garg,, Ayush Mangal

TL;DR
This paper explores how providing hints can enhance large language models' ability to solve complex math problems, inspired by human pedagogical methods, and evaluates their robustness to adversarial hints across diverse models and problems.
Contribution
It introduces a hint-based prompting technique for LLMs to improve mathematical reasoning and assesses its effectiveness and robustness compared to existing prompting methods.
Findings
Hints significantly improve LLM performance on math problems.
Models show sensitivity to adversarial hints, affecting accuracy.
Hint-based prompting outperforms one-shot and chain-of-thought methods.
Abstract
While state-of-the-art LLMs have shown poor logical and basic mathematical reasoning, recent works try to improve their problem-solving abilities using prompting techniques. We propose giving "hints" to improve the language model's performance on advanced mathematical problems, taking inspiration from how humans approach math pedagogically. We also test robustness to adversarial hints and demonstrate their sensitivity to them. We demonstrate the effectiveness of our approach by evaluating various diverse LLMs, presenting them with a broad set of problems of different difficulties and topics from the MATH dataset and comparing against techniques such as one-shot, few-shot, and chain of thought prompting.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsLegal Education and Practice Innovations · Mathematics, Computing, and Information Processing
MethodsSparse Evolutionary Training
