Dissecting Service Mesh Overheads

TL;DR

This paper introduces MeshInsight, a tool for systematically measuring service mesh overheads, revealing significant latency and resource impacts that depend on configuration and workload, and evaluating optimization effects.

Contribution

The paper presents MeshInsight, a novel decompositional approach and tool for quantifying service mesh overheads and analyzing the effects of configurations and optimizations.

Findings

Service meshes can cause up to 185% higher latency.

Resource consumption can increase by up to 92% CPU cores.

Overheads are highly dependent on configuration and workload.

Abstract

Service meshes play a central role in the modern application ecosystem by providing an easy and flexible way to connect different services that form a distributed application. However, because of the way they interpose on application traffic, they can substantially increase application latency and resource consumption. We develop a decompositional approach and a tool, called MeshInsight, to systematically characterize the overhead of service meshes and to help developers quantify overhead in deployment scenarios of interest. Using MeshInsight, we confirm that service meshes can have high overhead -- up to 185% higher latency and up to 92% more virtual CPU cores for our benchmark applications -- but the severity is intimately tied to how they are configured and the application workload. The primary contributors to overhead vary based on the configuration too. IPC (inter-process communication) and socket writes dominate when the service mesh operates as a TCP proxy, but protocol parsing dominates when it operates as an HTTP proxy. MeshInsight also enables us to study the end-to-end impact of optimizations to service meshes. We show that not all seemingly-promising optimizations lead to a notable overhead reduction in realistic settings.