Automatic and Efficient Customization of Neural Networks for ML Applications

TL;DR

This paper introduces ChameleonAPI, a framework that customizes neural network models for specific applications by analyzing their decision processes, leading to a 43% reduction in incorrect decisions without altering application code.

Contribution

It proposes a novel optimization framework that automatically creates application-specific neural network models for ML APIs, improving decision accuracy.

Findings

Reduces incorrect application decisions by 43%.

Analyzes 77 real-world applications to identify decision-critical API errors.

Provides a parser and loss function tailored to each application's decision process.

Abstract

ML APIs have greatly relieved application developers of the burden to design and train their own neural network models -- classifying objects in an image can now be as simple as one line of Python code to call an API. However, these APIs offer the same pre-trained models regardless of how their output is used by different applications. This can be suboptimal as not all ML inference errors can cause application failures, and the distinction between inference errors that can or cannot cause failures varies greatly across applications. To tackle this problem, we first study 77 real-world applications, which collectively use six ML APIs from two providers, to reveal common patterns of how ML API output affects applications' decision processes. Inspired by the findings, we propose ChameleonAPI, an optimization framework for ML APIs, which takes effect without changing the application source code. ChameleonAPI provides application developers with a parser that automatically analyzes the application to produce an abstract of its decision process, which is then used to devise an application-specific loss function that only penalizes API output errors critical to the application. ChameleonAPI uses the loss function to efficiently train a neural network model customized for each application and deploys it to serve API invocations from the respective application via existing interface. Compared to a baseline that selects the best-of-all commercial ML API, we show that ChameleonAPI reduces incorrect application decisions by 43%.