AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarios

TL;DR

AgentCoMa introduces a benchmark for evaluating LLMs on tasks requiring combined commonsense and mathematical reasoning, revealing significant performance gaps and model brittleness in mixed reasoning scenarios.

Contribution

This work presents a novel benchmark, AgentCoMa, specifically designed to test LLMs on combined commonsense and math reasoning tasks, highlighting current limitations and providing a platform for future improvements.

Findings

LLMs solve individual reasoning steps well but struggle with combined tasks.

Performance drops by ~30% when reasoning types are mixed.

Humans perform consistently well on the combined tasks.

Abstract

Large Language Models (LLMs) have achieved high accuracy on complex commonsense and mathematical problems that involve the composition of multiple reasoning steps. However, current compositional benchmarks testing these skills tend to focus on either commonsense or math reasoning, whereas LLM agents solving real-world tasks would require a combination of both. In this work, we introduce an Agentic Commonsense and Math benchmark (AgentCoMa), where each compositional task requires a commonsense reasoning step and a math reasoning step. We test it on 61 LLMs of different sizes, model families, and training strategies. We find that LLMs can usually solve both steps in isolation, yet their accuracy drops by ~30% on average when the two are combined. This is a substantially greater performance gap than the one we observe in prior compositional benchmarks that combine multiple steps of the same reasoning type. In contrast, non-expert human annotators can solve the compositional questions and the individual steps in AgentCoMa with similarly high accuracy. Furthermore, we conduct a series of interpretability studies to better understand the performance gap, examining neuron patterns, attention maps and membership inference. Our work underscores a substantial degree of model brittleness in the context of mixed-type compositional reasoning and offers a test bed for future improvement.