StackVault: Protection from Untrusted Functions

TL;DR

StackVault is a kernel-based system that protects sensitive stack data from unauthorized access by untrusted functions within the same process, enhancing security against data exfiltration attacks.

Contribution

It introduces a novel kernel module and LLVM extension to automatically enforce stack data protection with minimal performance overhead.

Findings

Effective in preventing unauthorized stack data access.

Incurs only up to 2.4% runtime overhead.

Validated on real-world applications like gRPC.

Abstract

Data exfiltration attacks have led to huge data breaches. Recently, the Equifax attack affected 147M users and a third-party library - Apache Struts - was alleged to be responsible for it. These attacks often exploit the fact that sensitive data are stored unencrypted in process memory and can be accessed by any function executing within the same process, including untrusted third-party library functions. This paper presents StackVault, a kernel-based system to prevent sensitive stack-based data from being accessed in an unauthorized manner by intra-process functions. Stack-based data includes data on stack as well as data pointed to by pointer variables on stack. StackVault consists of three components: (1) a set of programming APIs to allow users to specify which data needs to be protected, (2) a kernel module which uses unforgeable function identities to reliably carry out the sensitive data protection, and (3) an LLVM compiler extension that enables transparent placement of stack protection operations. The StackVault system automatically enforces stack protection through spatial and temporal access monitoring and control over both sensitive stack data and untrusted functions. We implemented StackVault and evaluated it using a number of popular real-world applications, including gRPC. The results show that StackVault is effective and efficient, incurring only up to 2.4% runtime overhead.

Figures

Tables

Table 1. Table I: Mapping from StackVault API to system calls

API	System Call(s)	Parameter	Comment
Untrusted function	start_protect and stop_protect		before and after call
Sensitive function	register_stack and unregister_stack	all=True	begin and end of function
Sensitive_Finegrained function	register_stack and unregister_stack	all=False	begin and end of function
Sensitive var	register_memory	readOnly=False	only when all=False
NotSensitive var	register_memory_exception	readOnly=False	only when all=True
WriteSensitive var	register_memory	readOnly=True	only when all=False
WriteSensitive var	register_memory_exception	readOnly=True	only when all=True
SensitivePointer var∗	register_memory	readOnly=False	for pointee object
WriteSensitivePointer var∗	register_memory	readOnly=True	for pointee object

Table 2. Table II: Application details

Application	Category	LoC	sensitive funcs	# of untrusted funcs	Third Parties
gRPC	Remote procedure call benchmark	6049	QpsWorker RunClientBody RunServerBody	2	gRPC
libcurl-xmlstream	File parser	103	main	14	libcurl, expat
libcurl-fileupload	File upload	41	main	6	libcurl
libcurl-htmltidy	File parser	83	main	20	libcurl, libtidy
minizip	File compression tool	520	main	3	zlib
miniunz	File compression tool	671	main	3	zlib

Table 3. Table III: Number of each StackVault related system calls invoked by the applications

	register _stack()	unregister _stack()	register_ memory()	register_ memory_ exception()	start_ protect()	stop_ protect()	Total	Total stack-based memory copied and cleared in bytes
gRPC	3	3	0	0	23	23	52	26,960
xmlstream	1	1	0	1	112	121	245	29,200
fileupload	1	1	0	2	10	10	24	5,824
htmltidy	1	1	1	147	2,053	2,053	4,255	2,187,680
minizip	1	1	0	6	11	11	30	32,114
miniunz	1	1	0	1	7	7	17	16,528

Table 4. Table IV: Comparison of executable file size(bytes) between the Native case and the StackVault case

Application	Native	StackVault	Overhead
gRPC	37,281,600	37,282,936	0.03%
xmlstream	13,416	13,472	0.42%
fileupload	12,768	12,824	0.44%
htmltidy	13,584	13,640	0.41%
minizip	81,664	81,792	0.16%
miniunz	77,200	77,336	0.18%

Full text

1 Introduction

One of the central trust assumptions of software systems is that a function can access data in the memory of another function in the same process. By exploiting this trust assumption, malicious functions can carry out "function-based data access attacks" to access sensitive data in the memory of other functions, thus compromising data security and enabling a channel for data exfiltration attacks. Such sensitive data can include protected health information (PHI) and sensitive personal information (SPI), which are required to be protected by privacy regulations such as GDPR [1] and HIPAA [2].

Function-based data access attacks can be enabled by malicious code or security bugs in functions in the code-base. Today, the use of third-party libraries and open source code in software services and products is widespread, and this has increased the use of untrusted functions included in application code [43, 42]. Untrusted functions may not have been subject to rigorous in-house software development and testing practices, thus they could contain security bugs. Such security bugs can be introduced from multiple sources. First, open source code repositories have been demonstrated to be susceptible to vulnerabilities [12], and an increased number of compromises have been observed in the recent past [14]. This makes it possible for security bugs to be introduced into open-source code with malicious intent. Second, inadvertent programming errors can also bring in such bugs. For example, Equifax recently blamed security holes in Apache Struts to be the reason behind the 147 Million user records data breach [37, 13]. In another example, Heartbleed occurred due to a security bug (inadvertent or malicious in nature) in the open source OpenSSL code [38, 50]. Third, the bugs may also be injected by malicious inside members of a software engineering team. Insiders may contribute new code or sabotage existing code in order to implant security bugs that allow access to sensitive data in the memory of other functions, even when there is no need-to-know [18] such data. It is reported that a Tesla employee had changed parts of the company’s manufacturing operating system code and sent "highly sensitive" company data to outside parties [11].

There are several works on security of stack memory [21, 27, 16, 48] as well as heap memory [41, 34, 32, 40, 46]. They primarily focus on attacks based on corrupting the contents of heap, or exposing/compromising the stack address and corrupting the contents of stack memory. However, to the best of our knowledge, data access isolation between functions in the same process has not been addressed in the literature so far. Shreds [22] and hardware architectures such as SGX [24] and SecureBlue++ [19] protect from outside-in memory access by external processes/threats; outside-in threat is outside of a process and goal is to inject/read data in the memory of the process. But "function-based data access attacks" enable inside-out threat, where the threat is inside the process and attacker goal is to exfiltrate data outside the process.

In this paper, we address the problem of how to protect stack-based data from function-based data access attacks. Our focus is to protect data on stacks and also the data in memory pointed to by stack variables (we call these stack-based data in this paper). Protecting the stack is a different problem from protecting the heap because stack frames are allocated in a systematic and managed manner and are transient by design. This makes it possible to devise a more efficient solution for protecting stack-based data compared to more general memory protection solutions. Further, in the context of using third-party libraries, the risk of data breaches and exfiltration can be significantly reduced by protecting just the stack-based data.

Our contributions: We present the design, implementation and experimental evaluation of our StackVault system that protects stack-based sensitive data from untrusted functions. This system provides the following guarantees: (1) while a sensitive function is executing, the sensitive stack-based data of the function should be preserved in a secure way (protection against corruption), and it should be protected from being read with malicious intent by untrusted functions (protection against data leakage); (2) when an untrusted function returns to its caller, the contents of its stackframe are cleared in order to prevent the "untrusted" contents to remain on stack of the thread/process, which can later be used to carry out/enable attacks such as a multi-stage attack; (3) when an untrusted function returns to a caller that is a sensitive function, the contents of the sensitive function’s stackframe are restored; (4) when a sensitive function returns to its caller, the contents of its stackframe are cleared. StackVault is designed to support sequential, nested and recursive function calls.

How StackVault works: StackVault introduces a notion of "unforgeable function identity" and a kernel based mechanism to protect the sensitive data on the stack. It consists of three components: a set of programming APIs for users to specify what needs to be protected in the source code, a compiler to compile the source code with StackVault protection, and a kernel module with StackVault related system calls to carry out data protection.

Our design of StackVault encourages a practice for more efficient and cleaner secure programming; a developer should manage the sensitive data on the stack instead of scattering the sensitive data across memory by using both stacks and heaps, which increases the attack surface. Using only heaps is not always possible because function calls rely on stacks.

2 Attack and Threat Model

Function-based Data Access Attacks. The underlying assumption in today’s programming models "trusts" that a function shall enforce a "need-to-know" policy whenever it accesses the data in the stack and heap of other functions in the code. However, with proliferation of unauthorized/un-regulated open source code and libraries, legacy software, as well as insider threats with access to enterprise software supply chain, such an implicit trust assumption may no longer be valid.

In a function-based data access attack, untrusted functions can access and steal sensitive data from the stack-frames and in the heap of other functions that have been called earlier during the program execution. The vulnerability exploited by attackers is – any function can access the stack or the heap of another function in the same process. Such access is allowed in almost all processes and is an implicit trust assumption in software systems.

Sensitive and Untrusted Functions. A function in a program can be of the following two kinds – "untrusted function" and "sensitive function"111Trusted-not-sensitive function: for the sake of completeness, there is another kind of function that is trusted but does not contain/process/produce sensitive data. Such functions are not directly relevant to this paper, so we do not discuss these kind of functions..

• •

Untrusted function: an untrusted function can be any function whose behavior is not fully controlled by a user. Third party libraries are one category of untrusted functions, since they are not fully written in-house and can be compromised by being downloaded from a fake website. In another example, when multiple companies are contributing code to the same project, a function developed by one company can be an untrusted function for another company that is using the function.

• •

Sensitive function: a function that is fully developed in-house and contains sensitive stack-based data, which can be either a variable or parameter allocated on the stack, or a pointer to heap data.

Example. C/C++ support pointers and pointer arithmetic that can be used by f2() against its stack base pointer and get access to f1(). Figure 1 displays the stack layout of two functions f1() and f2() when they are invoked in either a sequential or a nested manner. We assume that f1() is a function that has sensitive data on its stack, while f2() is an untrusted function. In the nested case, the untrusted function f2() is invoked within f1(). Since the sensitive data of f1() still resides on its stack during the execution of f2(), f2() can easily get these data if it is compromised. In the sequential case, f1() allocates two pieces of sensitive data $-$ password and key $-$ on its stack, but does not clear them before it returns. Thus, when the untrusted function f2() starts to run, it will be able to access these sensitive data on the stack of f1().

In our solution, we assume the OS kernel is trusted and has not been compromised. Note that if an attacker has compromised the kernel, StackVault may need to be implemented using a hardware root of trust or as a complete hardware system (which is out of the scope for this paper).

3 System Design

A system providing StackVault protection guarantees that all previously allocated memory that has been specified as sensitive will be inaccessible in the scope of an untrusted function. This goal is achieved by assuming a trusted kernel and appropriately hiding sensitive data in kernel buffers during execution of untrusted functions.

Figure 2 provides an overview of StackVault, which consists of three main components: a StackVault programming API, a Linux kernel module, and an LLVM C/C++ compiler extension. To protect an application with StackVault, programmers first use the programming API to denote the sensitive and untrusted functions in the application source code. Next, the source code is compiled using a modified LLVM compiler, which automatically injects StackVault system calls into the executable file. Then, during execution of the application, these system calls invoke the StackVault kernel module to protect sensitive stack-based data. The following sections describe in detail each of the three StackVault components.

3.1 StackVault Programming API

In our work, we rely on developers to explicitly specify untrusted functions and sensitive functions with stack-based data to protect. It is possible for an automatic or semi-automatic framework to be developed for this purpose, e.g. by using formal methods, analysis techniques, or heuristic rules. Such a framework is complementary to our work and can leverage our system by generating code with StackVault APIs appropriately inserted. The StackVault programming API provides the capability to:

(1)Specify untrusted functions:

Untrusted functions are specified by listing the function prototypes in a supplementary file called UntrustedList. The format is similar to declaring functions in C/C++ header files. For an untrusted third-party library, the header file defining the library functions can be directly used as the supplementary file.

(2)Specify sensitive functions:

Sensitive functions contain sensitive data (or references to sensitive data) in parameters or local variables. Like untrusted functions, they are specified using a supplementary file called SensitiveList. Alternatively, sensitive functions can be specified by an attribute directive inserted directly in the code at the point of the function definition, as follows:

__attribute((annotate(“StackVault_Sensitive”))) void foo(int x, …){…}

By default, the entire stack frame of a sensitive function is assumed to be a sensitive memory region. However, the programmer can use the annotation string StackVault_Sensitive_Finegrained to indicate that the entire stack frame should not be protected.

(3)Annotate individual parameters and local variables:

For a sensitive function, any of its parameter or local variable declarations can be prepended with an attribute directive to allow fine-grained protection and access control. For example, a single parameter in the stack frame can be specified to be sensitive, as follows:

__attribute((annotate(“StackVault_Sensitive_Finegrained”))) void foo(

__attribute((annotate(“StackVault_Sensitive”))) int x, …){…}

The attribute string StackVault_Sensitive specifies the corresponding variable as a sensitive one to be protected from being accessed in untrusted code, whereas the attribute string StackVault_NotSensitive specifies the corresponding variable as one that can be freely accessed. The attribute string StackVault_WriteSensitive specifies a variable to be read-only in untrusted code, which ensures that the value stored in the variable will not be overwritten by untrusted function calls.

(4)Annotate pointer type parameters and local variables:

For a sensitive variable of pointer type, the address stored in the pointer is protected. However, by default, the memory pointed to by the pointer is not protected. The annotation string StackVault_SensitivePointer is provided to indicate that both the pointer and the memory object pointed to are sensitive. Likewise, the annotation string StackVault_WriteSensitivePointer specifies that both the pointer and the memory object are read-only in untrusted code.

In some cases, the size of the memory object can be automatically determined based on the type of the pointer. Otherwise, the user has to explicitly provide the size, by appending it to the annotation string in the format StackVault_SensitivePointer_x, where x is the size of the memory object in bytes. This is useful for handling void pointers and pointer-based arrays.

In our work, we handle protection for only one-level pointer-based memory and assume that the memory objects are contiguous blocks of memory. In general, memory objects can be recursive pointer-based data structures, and there is ongoing work in the community to investigate how shape descriptors for arbitrary data structures can be efficiently specified[51]. However, for all the applications we considered in the context of StackVault, assuming one-level contiguous pointer objects is sufficient to enable effective protection.

Figure 3 illustrates the use of the StackVault API with an example code snippet. Figure 3(a) shows the original code where pwdgenerator is a sensitive function, since it has sensitive data such as passwd and id allocated on its stack. lib_func is a third-party untrusted function which needs to access the age variable on the stack frame of the sensitive function, but must not be allowed to access any other data in that stack frame. Figure 3(b) shows the modified code that uses the StackVault programming API to enable this protection.

We defined the programming API such that it provides features necessary to express all possible cases of protection requirements. However, most of cases in practice only need to specify untrusted functions and sensitive functions, and this can be done without any changes in the source code.

3.2 StackVault Kernel Module

The StackVault kernel module provides a set of system calls to specify what stack-based memory to protect and to determine when to protect. Protection is guaranteed for an executable binary generated with these system calls inserted at appropriate locations. On loading an application binary, the kernel scans the text section (using an ELF parser) to record the contiguous instruction address span for each named function in the code. These instruction address spans then serve as unforgeable function identities; i.e. the kernel can ascertain whether a given program counter (PC) register value during execution corresponds to the code of a specific function or not, without the possibility of any function masquerading as another. These function identities are reliable due to two reasons: first, the text sections in executable binaries are read-only and cannot be dynamically modified, and second, the value of PC registers cannot be modified by malicious user-level code.

The kernel runtime maintains four data structures: a Function identity mapping table, a RegisterList to record what is to be protected, a ProtectList to record the sequence of the protected calls, and a SaveBuffer to temporarily save sensitive data. These data structures are only available to kernel code, and cannot be accessed from user space.

The following system calls are used to specify what to protect:

(1)register_stack(bool all):

This call is to be invoked at the very beginning of each sensitive function. The boolean parameter determines whether the entire stack frame is to be protected. When invoked, this call automatically determines the identity of the calling function based on the current PC register value. It also determines the bounds of the stack frame for the calling function based on the calling convention (using rbp and rsp register values on x86 architecture). It then creates a new entry in the RegisterList, and records the function identity, address bounds of the stack frame, and the boolean parameter value passed in.

(2)register_memory(char *base, unsigned long len, bool readOnly):

This call is used to specify fine-grained regions of the stack frame that are sensitive and to be protected. It is also used to protect the memory object pointed to by a sensitive pointer variable on the stack. When invoked, this call first verifies that the identity of the calling function is the same as the function identity corresponding to the last register_stack call recorded in the RegisterList. If it does not match, an exception is flagged by the kernel, since a different identity indicates an illegal invocation. Otherwise, the kernel creates a new entry in the RegisterList, and records the parameters passed in, which are the base address, length in bytes of the memory region to be protected, and the type of protection (read-only access, or full protection).

(3)register_memory_exception(char *base, unsigned long len, bool readOnly):

This call is used to specify fine-grained regions of a sensitive stack frame that must be accessible from within an untrusted function. It is particularly useful when the address of a variable on the stack of a sensitive function is passed as a parameter to the untrusted function. As in the previous case, this call first verifies that the identity of the calling function is the same as the last register_stack call recorded in the RegisterList. It also verifies that the memory region defined by base and len falls within the current stack frame. If so, it creates a new entry in the RegisterList, and records the parameters passed in.

(4)unregister_stack(void):

This call is to be invoked right before a sensitive function returns. As in the previous case, this call first verifies that the identity of the calling function is the same as the last register_stack call recorded in the RegisterList. If so, it removes the RegisterList entry of the last register_stack call and all subsequent entries to it as well. It also clears the stack frame of the calling function, which ensures that untrusted functions invoked later in execution will not be able to access any leftover sensitive stack data.

The following system calls are used to determine when to protect:

(1)start_protect(void):

This call is to be invoked right before an untrusted function call. It enables protection for all registered sensitive memory regions by copying the sensitive data to kernel buffers. It also clears the sensitive memory regions (except when the memory region is allowed read-only access) to ensure that sensitive data is not leaked to untrusted functions. This call also determines the identity of the calling function and creates a new entry in the ProtectList recording the function identity and the index of the first free entry in the RegisterList.

Note that in the case of nested calls to start_protect, protection will already be in effect for sensitive data registered prior to the previous start_protect call, and this invocation of start_protect needs to process only the data newly registered since then. These RegisterList entries are processed in order, and Algorithm 1 gives details of how this is done.

(2)stop_protect(void):

This call is to be invoked right after an untrusted function call returns. On invocation, it first determines the identity of the calling function, and verifies that it is the same as the function identity recorded in the last entry in the ProtectList. If the identities do not match, an exception is flagged. Then it verifies that the index of the first free entry in the RegisterList matches the index recorded in the last entry in the ProtectList. If it does not match, then there are improper (possibly malicious) entries in the RegisterList added during execution of untrusted code, and an exception is flagged.

Otherwise, the kernel removes the last ProtectList entry. Then it uses the $SaveBuffer$ to restore data for all sensitive memory regions. This ensures that the application continues executing correctly and remains unaffected by any corrupt data that may have been written into sensitive areas by untrusted code. The restoration of stack-based data is done by processing RegisterList entries in order, analogous to the processing for start_protect calls. This procedure assumes that for a given sensitive function, there is no overlap between the memory objects referred to by register_memory and register_memory_exception calls. This constraint is automatically satisfied when the StackVault programming API and compiler are used to generate the executable code.

Algorithm 2 gives details of how the stack frame restoration is done. Note that the pseudocode shown here gives the logic of the algorithm; the actual implementation is optimized for the common case. Also note that data written to SaveBuffer is read back exactly once, and buffer management can free up the corresponding space on a read.

Figure 4 shows the example of Figure 3 with appropriate StackVault system calls inserted.

System Call Invocation in Assembly

Although StackVault kernel runtime calls are designed as system calls, they need to be invoked using inline assembly (asm in C/C++) and are identified by the system call number, instead of being invoked directly using the system call names. This is because whenever a StackVault system call is invoked, it refers to the instruction pointer (saved PC register value) to determine the identity of the function that issued the invocation, so that illegal invocations from untrusted functions can be detected. However, when system calls are invoked by their names, the instruction pointer obtained within the system call will always point to the same address in kernel code, regardless of the function invoking the system call. This is because the name of the system call is a wrapper function, and the real system call is invoked inside this wrapper. Therefore, if the system call is invoked by its name, the StackVault system call will not be able to determine the unforgeable identity of the function that invokes the system call.

3.3 StackVault Compiler Extension

The StackVault compiler automatically transforms code annotated with the StackVault programming API to executable binaries with appropriate StackVault system calls inserted. We modified the LLVM compiler release version 6.0.0 to implement this functionality. This compiler provides new command-line options: -qstackvault to enable the StackVault system, and -qstackvault_dir to specify the directory path containing the SensitiveList and UntrustedList files. When invoked, the compiler front-end processes these files and registers the identities of sensitive and untrusted functions.

Then, the code generation phase incorporates the following changes for StackVault processing:

Function calls: If the function being invoked matches the prototype of an untrusted function, then the compiler inserts a start_protect system call after the code setting up the function call (i.e. after the callee stack frame has been allocated and argument values have been copied, but before the jump to the callee function code). It also inserts a stop_protect system call in the code generated to process the return, right before the jump back to the caller code. Further, it checks all arguments of the untrusted function to see if any is of pointer type and is assigned the address of a local variable on the caller stack frame. For any such argument, it treats the corresponding local variable as StackVault_NotSensitive, and inserts a register_memory_exception system call if needed. 2. 2.

Function definitions: If the function being generated matches a sensitive function, then the compiler inserts a register_stack system call at the end of the function prologue, and an unregister_stack system call at the beginning of the function epilogue. 3. 3.

Local stack variables: Function parameters and local variables typically have space allocated to them on the function stack frame. When processing them, the compiler checks the parameter/variable definition to see if there is any StackVault annotation attached. If so, it inserts the appropriate register_memory or register_memory_exception system calls at the end of the function prologue.

Table I lists all the mappings from StackVault programming APIs to StackVault system calls. For all sensitive and untrusted functions, the compiler ensures that the function is not inlined. This allows the runtime verification to properly determine the function identity. Also, the -fno-omit-frame-pointer compiler flag is used to make sure the frame pointer register is always set up, so that the kernel runtime can properly determine the bounds of the function stack frame on a register_stack call.

StackVault is compiled with link-time optimization enabled, and all StackVault system calls inserted during compilation are inlined.

4 Security Analysis

In this section, we show how the StackVault system design makes it robust against adversarial manipulations that attackers may try to use. At runtime, the StackVault kernel module tracks corresponding pairs of register_stack and unregister_stack calls, and pairs of start_protect and stop_protect calls. It ensures that these calls are invoked in a legal and orderly manner by using runtime verification checks and relying on unforgeable function identities. Thus, even when an attacker is familiar with the StackVault system calls, the design prevents evading or misusing StackVault security protection.

First, the attacker may try to evade StackVault protection by invoking stop_protect in untrusted functions to illegally restore the protected stack and steal sensitive information from there. StackVault defends against this attack by checking if the function invoking stop_protect is the same as the function that most recently invoked start_protect. If not, then this invocation will raise an exception, and a warning will be given to alert users that their code is under attack. Even if the attacker manages to hijack the program control flow and branch to the execution of the stop_protect call in the appropriate function, any return to untrusted function code will require crossing another start_protect call boundary, and the attack will fail.

The attacker may also try to evade protection by using the register_memory_exception call in untrusted functions to void protection for specific memory regions. StackVault defends against this attack by checking if the memory region defined by the parameters of a register_memory_exception falls within the stack frame of the current function. As a result, this call can be used by a function to enable access to its own stack frame only, and it cannot affect any other (possibly sensitive) memory regions.

Second, the attacker may try to misuse StackVault protection by invoking register_stack or register_memory in untrusted functions to illegally restrict regular code from accessing memory regions that they should have access to. StackVault detects such misuse on return from untrusted functions, when stop_protect is invoked and the runtime checks if the number of entries in the kernel’s RegisterList is the same as it was when the corresponding start_protect was invoked. This check detects any improper RegisterList entries leftover from calls made in untrusted functions, and flags a warning to the user.

Note that the function identity checking prevents misuse of unregister_stack calls in untrusted functions. The unforgeable function identities used in StackVault cannot be subverted by a user level program and are a key component for enforcing protection. StackVault makes effective use of the PC register (for function identities) and stack register (for determining stack frame boundaries) as defined in the architecture and system-level API. StackVault also assumes a trusted kernel with privileged access to some memory buffers, and a cooperating compiler that does not inline any untrusted functions.

5 Evaluation

In this section, we first measure the overheads of StackVault by comparing the overall execution time of different applications while running with and without StackVault. Then, we zoom into more detailed StackVault runtime statistics to break down the overheads and investigate various factors that could have different impact on the overheads. Finally, we evaluate how StackVault affects the executable file size and the compilation time .

5.1 Experimental Setup

StackVault is evaluated using the following applications: gRPC, xmlstream, fileupload, htmltidy, minizip, and miniunz. In each of the applications, there is sensitive data allocated on the function stack, while the third party library calls are invoked in the same function. Therefore, such library calls could access the sensitive data if not being protected. The following two scenarios are compared in each case:

• $\bullet$

Native. The application is compiled with original LLVM; thus it is running without StackVault capabilities.

• $\bullet$

StackVault. Given a list of sensitive and untrusted functions, the application is compiled with the extended LLVM for StackVault, and it is running with StackVault protections.

All the experiments are conducted on an Intel Xeon E5640 server with 16 2.67GHz CPUs, 32 GB memory, and 500GB disk storage. Ubuntu 16.04 with Linux kernel 4.4.98 is used as the operating system. All the reported results are averaged over 5 runs. The details of each application used in the experiments are as follows:

• $\bullet$

gRPC[6]. gRPC is an open source remote procedure call system initially developed at Google. It is widely used to establish communications among different components in low latency, highly scalable, distributed systems. In this paper, we use a gRPC benchmark which runs 2 processes called QPS workers that act as the gRPC client and server, as well as one driver process that sets those workers up to run a specific test scenario. The driver sends the configuration parameters to the workers and reports the resulting statistics after the scenario is complete.

• $\bullet$

xmlstream[10]. This application uses the libcurl library to download an XML file from a given URL, and then parses this file via the streaming Expat parser. We run the xmlstream by downloading a 1MB XML file and parsing it.

• $\bullet$

fileupload[5]. This application uses the libcurl library to upload a file to a given URL. We run the fileupload to upload a 64MB file from one local directory to another local one.

• $\bullet$

htmltidy[7]. This application downloads an HTML document using the libcurl library and parses the document using the libtidy library. We run the htmltidy to download the html page from www.google.com and parse it.

• $\bullet$

minizip[9]. This application creates a compressed file from a normal file or directory. We run the minizip to compress a folder with 5 image files, which are in total 146MB.

• $\bullet$

miniunz[8]. This application uncompresses a compressed file. We run the miniunz to uncompresses the files compressed by minizip.

Table II shows the names of sensitive functions and number of untrusted functions in each application. The untrusted functions are from the Third Party libraries listed in the last column of the table. As discussed before, since the source code of such third party libraries are usually not written by the developers of the applications, these developers cannot have full control over the behavior of the functions in such libraries. The sensitive functions are the ones in the applications that have sensitive data on the stack. In practice, it highly depends on the developers of the applications to specifically define which data is sensitive. Taking the gRPC benchmark for an example, the functions RunClientBody() and RunServerBody() allocate sensitive data ClientArgs and ServerArgs on the stack. At the same time, a gRPC library call gpr_log() is frequently invoked by these two functions. In order to prevent gpr_log() from illegally accessing the sensitive data on the stack, RunClientBody() and RunServerBody() are selected as sensitive functions, while gpr_log() is selected as one of the untrusted functions. Similarly, main() in libcurl-fileupload is selected as a sensitive function, since it allocates a sensitive variable file_info on the stack, while also invoking many third party library calls such as curl_easy_getinfo(), which should not be allowed to access the file_info.

5.2 Execution time overhead

In this subsection, we examine StackVault’s impact on application performance. We instrument applications with gettimeofday() to obtain execution times in microseconds for each application in the native case and the StackVault case respectively.

The normalized execution time of each application is displayed in Figure 6. It shows that the execution time overhead is negligible for all the applications. For example, compared with the Native case, the execution time of fileupload increases 1.2% in the StackVault case, while that of minizip and miniunz increases 0.5% and 0.4% respectively. This indicates that even though additional system calls need to be invoked by the application in order to interact with StackVault, the overhead incurred by such system calls is small.

We zoom into the overhead to understand where it comes from and which part contributes most to such overhead. Basically, the additional execution time consists of two parts: first, since the application needs to invoke the StackVault specific system calls, there is overhead in context switching between user space and kernel space; the second part is the time spent inside each StackVault related system call. Figure 6 displays the normalized time spent on each component. We have two observations. First, for some applications, compared with the time spent inside the StackVault system calls, the context switch dominates the overhead. For instance, in gRPC and fileupload, 99% and 95% of the StackVault overhead is due to the context switch. Second, the distribution between the context switch overhead and the StackVault specific system calls overhead varies among different applications. Comparing minizip and xmlstream, although the context switch overhead exceeds the system call overhead for both applications, the context switch occupies 97% of the total overhead in minizip while it occupies 67% in htmltidy.

To explain such differences, we measure how frequently StackVault system calls are invoked in each application, and the results are displayed in Table III. There are a few interesting observations. First, xmlstream and htmltidy have the most StackVault system calls. This is consistent with the results in Figure 6, which show that xmlstream and htmltidy are the two applications that incur the most execution time overhead - 1.9% and 2.4% respectively. Second, most of the StackVault system calls invoked by xmlstream and htmltidy are start_protect() and stop_protect(). Recall that in Figure 6, xmlstream and htmltidy are the two applications that have the highest percentage of StackVault system call overhead; the overheads of start_protect() and stop_protect() are higher than their context switch cost, while that of the other StackVault system calls are less than the context switch overhead. Therefore, the more these two calls are invoked, the higher the system call overhead in terms of percentage. The results also indicate that on average, a context swtich between the user application and a StackVault system call takes 7 microseconds. In addition, the overhead of the system calls themselves depends on many other factors. For example, a larger stack size could bring more overhead since more data needs to be copied between the user stack and the kernel buffer.

We further divide the overhead of the StackVault system calls into each function, and the results are shown in Figure 7. It can be observed that in most cases, start_protect() and stop_protect() incur the most overhead among all the StackVault specific system calls. These two system calls are the most sophisticated ones among all the six. Both of them need to copy data between the user stack and the kernel buffer, clear the user stack or free the kernel buffer, and refer to the memory areas registered by register_memory() and register_memory_exception() to make sure these areas are properly handled. Since a sensitive function can include multiple untrusted functions, start_protect() and stop_protect() can be invoked many times between a single pair of register_stack() and unregister_stack() calls. For each application, stop_protect() incurs more overhead than start_protect(). Also, unregister_stack() incurs slightly more overhead than register_stack(). This is because unregister_stack() needs to clear the stack memory to make sure a sensitive function does not leave any sensitive data on the stack after it finishes running.

5.3 Executable size overhead

We compared the sizes of executable files between the Native case and the StackVault case. The results show that although the StackVault specific system calls are automatically inserted into the executable files when being compiled in the StackVault case, we see very little overhead in the executable file size. This is because compared with the number of instructions in the executable file, the number of inserted StackVault specific system calls is relatively small, which is negligible in this set of experiments.

Table IV shows that the increased executable file size due to StackVault is within 0.5%. Taking xmlstream for instance, with StackVault its executable file size increases 0.42% from 13,416 bytes to 13,472 bytes. Since the number of inserted StackVault system calls is small compared to the size of application code, the larger the executable file is the less the relative overhead. This is demonstrated by gRPC, where the overhead is only 0.03%.

5.4 Compilation time overhead

Figure 8 compares the compilation time of each application between the Native and the StackVault cases. The compilation overhead is significantly high (about 1.24 times longer) for the smaller applications (xmlstream, fileupload, and htmltidy), but it becomes much smaller for larger application sizes, taking about 0.47, 0.17, and 0.22 times longer for gRPC, minizip, and miniunz, respectively. The StackVault system calls are currently implemented as a static library, and the compiler implementation uses link-time optimization to inline these calls. This contributes significant compilation overhead, which can be optimized away by having the compiler directly generate the inline assembly instructions for each StackVault system call.

6 Discussion

The StackVault design handles recursive and nested functions for any ordering of sensitive and untrusted function calls. It also handles untrusted functions that are part of dynamically loaded libraries. If an untrusted function is invoked using a function pointer, the StackVault implementation is currently constrained to only handle them when the pointer can be resolved at compile time or link time. However, in general it is possible to use runtime code specialization to handle all calls invoked using function pointers.

Protection for multithreaded applications. In the presence of multiple threads, each thread has its own stack, but a thread can still access the stack-based data of another thread in the same process [4]. A simple solution for the StackVault design to provide protection from untrusted function calls across threads is to use locks and priority queues to ensure that no new sensitive function begins execution concurrently with the execution of an untrusted function. This can be implemented with a guarantee of forward progress as long as untrusted functions do not have blocking dependencies on application code. This is a reasonable assumption for untrusted functions that are third-party library functions or reusable API calls. This simple solution for multithreaded protection will add slight overhead to register_stack, start_protect, and stop_protect calls. More sophisticated solutions that allow higher application concurrency are possible, but have to be balanced with the extra implementation overhead.

Vulnerabilities of other programming languages and Systems. Although the current implementation of StackVault is based on C and C++, the mechanism introduced by StackVault can be applied to many other languages. Any programming model, language or runtime that relies on the assumption that "a function can access data in the memory of another function in the same process" is vulnerable to function-based data access attacks. A programming language is vulnerable to the function-based data access attacks if it satisfies one of the following criteria:

• •

It allows an untrusted function to directly or indirectly access the stack-based data of another function that has been called earlier.

• •

It has a facility that allows functions written in C/C++ (languages that support pointers and pointer arithmetic) to be invoked, even if the language itself does not support pointers/pointer arithmetic (such as Java/Python).

• •

It supports debugging APIs and tools to access/read call stack contents.

• •

It supports reflection that allows the program to introspect itself during execution.

Figure 9 presents the types of programming languages and systems that are vulnerable to function-based data access attacks.

There are other systems and programming paradigms that can benefit from StackVault, e.g. the micro-service paradigm. Micro-services support API calls, which are essentially function invocations in the form of remote procedure calls (over REST, SOAP). When such services receive parameters for the call, they use stack-frames to manage such data. The micro-services widely use third-party libraries in their implementation. Functions in those libraries (untrusted functions) when called from the API method implementation would allow these functions to access client-supplied data on stack-frames and/or in heaps pointed to by the stack-based variables.

7 Related Work

Stack based attacks. Due to the highly predictable layout of the stack memory, stack based attacks have existed for a long time, in which the most common one is buffer overflow [20]. For example, StackGuard [25] proposes two techniques to overcome the buffer overflow vulnerability. One is putting a canary word right besides each return address on the stack, so that the modification of the return address can be detected by checking whether the canary word has been changed. This idea has also been incorporated in the GCC [3] compiler. The other technique takes advantage of the debug registers to monitor the stack memory that stores the return address and triggers an exception once any return address has been rewritten. CRED [44] introduces a C range error detector, which allows programs to access out-of-bounds addresses that do not result in buffer overflows. Other stack protection approaches such as ASLR [45] and StackArmor [21] use randomization to make it difficult for the attackers to guess where the target stack frame is. The shadow stack [47, 27] was invented to protect return addresses on the stack from being tampered with. In this scheme, a shadow stack is maintained in parallel with the original stack, which is used to ensure the integrity of the address.

Control-Flow Integrity (CFI) based approaches [16, 26, 52, 15] are also designed to protect stack-based buffer overflows. Such approaches first construct a Control-Flow Graph (CFG) using source code analysis, binary analysis, or execution profiling. Then, software execution will be dictated to follow the CFG, so that a compromised execution path caused by buffer overflow will be prevented. Also data leakages may happen via uninitialized reads. In this case, an attacker can get the stack data via reading uninitialized stack variables. [39] and [36] are two recent efforts that solve this problem by explicitly initializing each local variable after it is allocated on the stack.

Strackx et al. [48] show how stack overreads can be used to carry out stack overflow attacks even in the presence of canaries and ASLR. The paper also talks about how stack overreads can also be used to override protections offered by memory obfuscation [17] and instruction set randomization [33]. Kundu and Bertino [35] presented how placement new in C++ can be exploited for carrying out buffer overflow as well as function pointer subterfuge attacks.

Memory data leakage. There is a long line of research on preventing memory data leakage. For instance, Shreds [22] protects the sensitive information in private memory by using the memory domain features in ARM CPU and [49] explores the trust issues in multithreaded applications such as MemCached. [31] finds that DRAM can retain its data for several seconds after it is powered off and removed from the motherboard and briefly discusses several solutions to these attacks, such as changing the architecture of the DRAM to make it lose state more quickly. SWIPE [30, 29] takes advantage of static analysis to erase the sensitive data at the earliest time. [23] presents a secure deallocation strategy to reduce the life cycle of the sensitive information in memory. Vanish [28] aims at creating self-destruct data that can automatically vanish when it is no longer useful. All of these efforts try to reduce the probability of data leakage by reducing its lifetime. However, sensitive data can still be leaked within its lifetime.

Hardware protection. SGX [24] hardware protects a portion of an application and data processed from threats outside the SGX enclave. SecureBlue++ [19] supports protection of a portion or the entire application or VM and the associated data from threats outside the application runtime. In contrast, the threat we are addressing in this paper is inside-out, that is the data can be accessed and exfiltrated from inside the application to outside.

In comparison, StackVault differs from the above mentioned works in that it identifies and reduces the vulnerabilities for stack-based attacks due to no isolation support among the functions within the same process. Moreover, existing memory isolation domains are at the process level, which are too coarse to prevent in-process function level illegal memory access.

8 Conclusions and Future Work

Untrusted third party libraries and malicious insider actors are becoming a significant threat leading to data leakage and exfiltration attacks. In this paper, we have described how function-based data access attacks can be enabled by malicious code in untrusted functions originating from such threats. We presented the StackVault system to protect stack-based sensitive data from being maliciously accessed by untrusted functions in the same process. StackVault introduces a set of programmer APIs, compiler extensions, and system calls that can be used to protect sensitive data during execution of untrusted functions and clear out the sensitive data from memory as soon as the sensitive function returns. StackVault uses a novel notion of unforgeable function identity in order to ensure that it will not be abused, subverted or spoofed by malicious functions. We have provided a security analysis and experimental evaluation of StackVault using popular real world applications such as gRPC. The results show that StackVault assures sufficient security guarantees while being highly efficient – it incurs very low overhead on the execution time of applications. StackVault can be used to enhance compliance with GDPR and HIPAA by protecting sensitive personal and healthcare data.

In the future, we are planning to develop schemes to protect the contents of heaps from function-based data access attacks (heap contents that are referred to by any stack variable are already protected by StackVault as presented in this paper). Another future direction is to evaluate different security solutions for multi-threaded applications.