TinyAgent: Function Calling at the Edge
Lutfi Eren Erdogan, Nicholas Lee, Siddharth Jha, Sehoon Kim, Ryan, Tabrizi, Suhong Moon, Coleman Hooper, Gopala Anumanchipalli, Kurt Keutzer,, Amir Gholami

TL;DR
TinyAgent is a framework for training small, efficient language models capable of function calling, enabling edge deployment of agentic systems that can perform complex tasks locally, rivaling larger cloud-based models.
Contribution
The paper introduces TinyAgent, a novel approach for fine-tuning small language models with function calling capabilities suitable for edge deployment, including new datasets and inference techniques.
Findings
TinyAgent models outperform larger models like GPT-4-Turbo in function calling.
Edge-deployed TinyAgent can execute user commands on MacBook effectively.
The framework enables real-time, local agentic systems comparable to cloud-based solutions.
Abstract
Recent large language models (LLMs) have enabled the development of advanced agentic systems that can integrate various tools and APIs to fulfill user queries through function calling. However, the deployment of these LLMs on the edge has not been explored since they typically require cloud-based infrastructure due to their substantial model size and computational demands. To this end, we present TinyAgent, an end-to-end framework for training and deploying task-specific small language model agents capable of function calling for driving agentic systems at the edge. We first show how to enable accurate function calling for open-source models via the LLMCompiler framework. We then systematically curate a high-quality dataset for function calling, which we use to fine-tune two small language models, TinyAgent-1.1B and 7B. For efficient inference, we introduce a novel tool retrieval method…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsContext-Aware Activity Recognition Systems
