Knowledge Grafting: A Mechanism for Optimizing AI Model Deployment in Resource-Constrained Environments

TL;DR

This paper introduces knowledge grafting, a novel method for optimizing large AI models for resource-limited devices by transferring key features from a big donor model to a smaller, efficient rootstock model, improving accuracy and reducing size.

Contribution

The paper presents a new knowledge grafting mechanism that effectively compresses AI models while enhancing their generalization, enabling deployment in resource-constrained environments.

Findings

Model size reduced by 88.54% from 64.39 MB to 7.38 MB

Rootstock model achieves 89.97% validation accuracy, surpassing the donor

Lower validation loss of 0.2976 indicates improved generalization

Abstract

The increasing adoption of Artificial Intelligence (AI) has led to larger, more complex models with numerous parameters that require substantial computing power -- resources often unavailable in many real-world application scenarios. Our paper addresses this challenge by introducing knowledge grafting, a novel mechanism that optimizes AI models for resource-constrained environments by transferring selected features (the scion) from a large donor model to a smaller rootstock model. The approach achieves an 88.54% reduction in model size (from 64.39 MB to 7.38 MB), while improving generalization capability of the model. Our new rootstock model achieves 89.97% validation accuracy (vs. donor's 87.47%), maintains lower validation loss (0.2976 vs. 0.5068), and performs exceptionally well on unseen test data with 90.45% accuracy. It addresses the typical size vs performance trade-off, and enables deployment of AI frameworks on resource-constrained devices with enhanced performance. We have tested our approach on an agricultural weed detection scenario, however, it can be extended across various edge computing scenarios, potentially accelerating AI adoption in areas with limited hardware/software support -- by mirroring in a similar manner the horticultural grafting enables productive cultivation in challenging agri-based environments.