Automated Profile-Guided Replacement of Data Structures to Reduce Memory Allocation

TL;DR

This paper introduces an automated method integrated into GraalVM Native Image that profiles Java applications to replace data structures with more memory-efficient versions, reducing overall memory usage.

Contribution

It presents a novel automated profiling and replacement approach for data structures in Java, integrated into an AOT compiler, to optimize memory consumption.

Findings

Memory usage reduced by up to 13.85% in benchmarks

Average memory reduction of 1.63% in standard benchmarks

Average memory reduction of 2.94% in microservice benchmarks

Abstract

Data structures are a cornerstone of most modern programming languages. Whether they are provided via separate libraries, built into the language specification, or as part of the language's standard library -- data structures such as lists, maps, sets, or arrays provide programmers with a large repertoire of tools to deal with data. Moreover, each kind of data structure typically comes with a variety of implementations that focus on scalability, memory efficiency, performance, thread-safety, or similar aspects. Choosing the *right* data structure for a particular use case can be difficult or even impossible if the data structure is part of a framework over which the user has no control. It typically requires in-depth knowledge about the program and, in particular, about the usage of the data structure in question. However, it is usually not feasible for developers to obtain such information about programs in advance. Hence, it makes sense to look for a more automated way for optimizing data structures. We present an approach to automatically replace data structures in Java applications. We use profiling to determine allocation-site-specific metrics about data structures and their usages, and then automatically replace their allocations with customized versions, focusing on memory efficiency. Our approach is integrated into GraalVM Native Image, an Ahead-of-Time compiler for Java applications. By analyzing the generated data structure profiles, we show how standard benchmarks and microservice-based applications use data structures and demonstrate the impact of customized data structures on the memory usage of applications. We conducted an evaluation on standard and microservice-based benchmarks, in which the memory usage was reduced by up to 13.85 % in benchmarks that make heavy use of data structures. While others are only slightly affected, we could still reduce the average memory usage by 1.63 % in standard benchmarks and by 2.94 % in microservice-based benchmarks. We argue that our work demonstrates that choosing appropriate data structures can reduce the memory usage of applications. While acknowledge that our approach does not provide benefits for all kinds of workloads, our work nevertheless shows how automated profiling and replacement can be used to optimize data structures in general. Hence, we argue that our work could pave the way for future optimizations of data structures.