SYSPART: Automated Temporal System Call Filtering for Binaries

TL;DR

SYSPART is an automated binary analysis tool that accurately identifies server phases and generates system-call filters, enhancing security without requiring source code access or manual phase detection.

Contribution

It introduces a novel static-dynamic analysis algorithm for binary-only server programs to automatically identify phases and generate precise system-call filters.

Findings

Outperforms prior binary-only approaches in identifying server phases

Generates accurate system-call filters for security

Performs comparably to source-code based methods

Abstract

Restricting the system calls available to applications reduces the attack surface of the kernel and limits the functionality available to compromised applications. Recent approaches automatically identify the system calls required by programs to block unneeded ones. For servers, they even consider different phases of execution to tighten restrictions after initialization completes. However, they require access to the source code for applications and libraries, depend on users identifying when the server transitions from initialization to serving clients, or do not account for dynamically-loaded libraries. This paper introduces SYSPART, an automatic system-call filtering system designed for binary-only server programs that addresses the above limitations. Using a novel algorithm that combines static and dynamic analysis, SYSPART identifies the serving phases of all working threads of a server. Static analysis is used to compute the system calls required during the various serving phases in a sound manner, and dynamic observations are only used to complement static resolution of dynamically-loaded libraries when necessary. We evaluated SYSPART using six popular servers on x86-64 Linux to demonstrate its effectiveness in automatically identifying serving phases, generating accurate system-call filters, and mitigating attacks. Our results show that SYSPART outperforms prior binary-only approaches and performs comparably to source-code approaches.