Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestration

TL;DR

Octopus introduces a new multimodal reasoning framework that enables autonomous exploration and dynamic capability selection, significantly improving performance across diverse tasks by mimicking human-like reasoning abilities.

Contribution

It defines six core reasoning capabilities, organizes a comprehensive benchmark, and demonstrates a system that autonomously orchestrates these capabilities for improved multimodal reasoning.

Findings

Achieves state-of-the-art performance on Octopus-Bench tasks

Demonstrates effective autonomous capability exploration and selection

Highlights importance of capability coordination in multimodal reasoning

Abstract

Existing multimodal reasoning models and frameworks suffer from fundamental architectural limitations: most lack the human-like ability to autonomously explore diverse reasoning pathways-whether in direct inference, tool-driven visual exploration, programmatic visual manipulation, or intrinsic visual imagination. Consequently, they struggle to adapt to dynamically changing capability requirements in real-world tasks. Meanwhile, humans exhibit a complementary set of thinking abilities when addressing such tasks, whereas existing methods typically cover only a subset of these dimensions. Inspired by this, we propose Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestration, a new paradigm for multimodal agentic reasoning. We define six core capabilities essential for multimodal reasoning and organize a comprehensive evaluation benchmark, Octopus-Bench, accordingly. Octopus is capable of autonomously exploring during reasoning and dynamically selecting the most appropriate capability based on the current state. Experimental results show that Octopus achieves the best performance on the vast majority of tasks in Octopus-Bench, highlighting the crucial role of capability coordination in agentic multimodal reasoning.