Energy-Efficient AI over a Virtualized Cloud Fog Network

TL;DR

This paper proposes an MILP-based method for optimal placement of DNN inference models in a cloud fog network to reduce energy consumption and latency, addressing the limitations of centralized cloud data centers.

Contribution

It introduces a novel formulation of DNN placement as a network embedding problem in a CFN architecture, balancing processing and networking for energy efficiency.

Findings

Significant energy savings compared to baseline CDC deployment

Effective trade-offs between processing and networking in fog nodes

Improved latency performance in edge computing scenarios

Abstract

Deep Neural Networks (DNNs) have served as a catalyst in introducing a plethora of next-generation services in the era of Internet of Things (IoT), thanks to the availability of massive amounts of data collected by the objects on the edge. Currently, DNN models are used to deliver many Artificial Intelligence (AI) services that include image and natural language processing, speech recognition, and robotics. Accordingly, such services utilize various DNN models that make it computationally intensive for deployment on the edge devices alone. Thus, most AI models are offloaded to distant cloud data centers (CDCs), which tend to consolidate large amounts of computing and storage resources into one or more CDCs. Deploying services in the CDC will inevitably lead to excessive latencies and overall increase in power consumption. Instead, fog computing allows for cloud services to be extended to the edge of the network, which allows for data processing to be performed closer to the end-user device. However, different from cloud data centers, fog nodes have limited computational power and are highly distributed in the network. In this paper, using Mixed Integer Linear Programming (MILP), we formulate the placement of DNN inference models, which is abstracted as a network embedding problem in a Cloud Fog Network (CFN) architecture, where power savings are introduced through trade-offs between processing and networking. We study the performance of the CFN architecture by comparing the energy savings when compared to the baseline approach which is the CDC.