HGMF: A Hierarchical Gaussian Mixture Framework for Scalable Tool Invocation within the Model Context Protocol

TL;DR

HGMF is a hierarchical probabilistic framework that improves large-scale tool invocation accuracy and efficiency in LLMs by clustering and filtering tools based on semantic similarity, reducing noise and computational costs.

Contribution

The paper introduces HGMF, a novel hierarchical Gaussian Mixture Model-based method for scalable and accurate tool selection in large tool libraries for LLMs.

Findings

Significantly improves tool selection accuracy.

Reduces inference latency in large-scale tool invocation.

Effective in handling large, hierarchically-structured tool libraries.

Abstract

Invoking external tools enables Large Language Models (LLMs) to perform complex, real-world tasks, yet selecting the correct tool from large, hierarchically-structured libraries remains a significant challenge. The limited context windows of LLMs and noise from irrelevant options often lead to low selection accuracy and high computational costs. To address this, we propose the Hierarchical Gaussian Mixture Framework (HGMF), a probabilistic pruning method for scalable tool invocation. HGMF first maps the user query and all tool descriptions into a unified semantic space. The framework then operates in two stages: it clusters servers using a Gaussian Mixture Model (GMM) and filters them based on the query's likelihood. Subsequently, it applies the same GMM-based clustering and filtering to the tools associated with the selected servers. This hierarchical process produces a compact, high-relevance candidate set, simplifying the final selection task for the LLM. Experiments on a public dataset show that HGMF significantly improves tool selection accuracy while reducing inference latency, confirming the framework's scalability and effectiveness for large-scale tool libraries.