Interprocess Communication in FreeBSD 11: Performance Analysis

TL;DR

This paper analyzes the performance of pipes and sockets in FreeBSD 11, revealing that pipes outperform sockets and that buffer size significantly impacts IPC efficiency, with detailed investigation and validation of experimental results.

Contribution

It provides a comprehensive performance analysis of IPC mechanisms in FreeBSD 11, including benchmarking, hardware analysis, and insights into buffer size effects.

Findings

Pipes outperform sockets by 63% on average.

Larger buffers improve performance up to 32-64 KiB, then decline.

Probe effects are validated to ensure result accuracy.

Abstract

Interprocess communication, IPC, is one of the most fundamental functions of a modern operating system, playing an essential role in the fabric of contemporary applications. This report conducts an investigation in FreeBSD of the real world performance considerations behind two of the most common IPC mechanisms; pipes and sockets. A simple benchmark provides a fair sense of effective bandwidth for each, and analysis using DTrace, hardware performance counters and the operating system's source code is presented. We note that pipes outperform sockets by 63% on average across all configurations, and further that the size of userspace transmission buffers has a profound effect on performance - larger buffers are beneficial up to a point (~32-64 KiB) after which performance collapses as a result of devastating cache exhaustion. A deep scrutiny of the probe effects at play is also presented, justifying the validity of conclusions drawn from these experiments.