ParaView-MCP: An Autonomous Visualization Agent with Direct Tool Use

TL;DR

ParaView-MCP introduces an autonomous agent that integrates large language models with ParaView, enabling natural language and visual interaction, simplifying complex visualization tasks, and supporting intelligent decision-making in scientific visualization.

Contribution

The paper presents ParaView-MCP, a novel system combining multimodal large language models with ParaView via the Model Context Protocol for seamless, intelligent visualization interactions.

Findings

Enables natural language interaction with ParaView

Supports visual feedback for recreating and updating visualizations

Facilitates cross-application collaboration

Abstract

While powerful and well-established, tools like ParaView present a steep learning curve that discourages many potential users. This work introduces ParaView-MCP, an autonomous agent that integrates modern multimodal large language models (MLLMs) with ParaView to not only lower the barrier to entry but also augment ParaView with intelligent decision support. By leveraging the state-of-the-art reasoning, command execution, and vision capabilities of MLLMs, ParaView-MCP enables users to interact with ParaView through natural language and visual inputs. Specifically, our system adopted the Model Context Protocol (MCP) - a standardized interface for model-application communication - that facilitates direct interaction between MLLMs with ParaView's Python API to allow seamless information exchange between the user, the language model, and the visualization tool itself. Furthermore, by implementing a visual feedback mechanism that allows the agent to observe the viewport, we unlock a range of new capabilities, including recreating visualizations from examples, closed-loop visualization parameter updates based on user-defined goals, and even cross-application collaboration involving multiple tools. Broadly, we believe such an agent-driven visualization paradigm can profoundly change the way we interact with visualization tools. We expect a significant uptake in the development of such visualization tools, in both visualization research and industry.