Surface Compression Using Dynamic Color Palettes

TL;DR

This paper introduces Dynamic Color Palettes (DCP), a hardware-based lossless compression scheme for framebuffer surfaces that significantly reduces off-chip memory bandwidth for mobile graphics, especially in UI and 2D applications.

Contribution

The paper proposes DCP, a novel dynamic palette construction method that exploits inter-frame coherence to improve framebuffer surface compression rates in mobile graphics.

Findings

DCP achieves 91% higher compression for UI surfaces.

DCP achieves 20% higher compression for 2D surfaces.

Hybrid schemes with DCP outperform previous methods by up to 161%.

Abstract

Off-chip memory traffic is a major source of power and energy consumption on mobile platforms. A large amount of this off-chip traffic is used to manipulate graphics framebuffer surfaces. To cut down the cost of accessing off-chip memory, framebuffer surfaces are compressed to reduce the bandwidth consumed on surface manipulation when rendering or displaying. In this work, we study the compression properties of framebuffer surfaces and highlight the fact that surfaces from different applications have different compression characteristics. We use the results of our analysis to propose a scheme, Dynamic Color Palettes (DCP), which achieves higher compression rates with UI and 2D surfaces. DCP is a hardware mechanism for exploiting inter-frame coherence in lossless surface compression; it implements a scheme that dynamically constructs color palettes, which are then used to efficiently compress framebuffer surfaces. To evaluate DCP, we created an extensive set of OpenGL workload traces from 124 Android applications. We found that DCP improves compression rates by 91% for UI and 20% for 2D applications compared to previous proposals. We also evaluate a hybrid scheme that combines DCP with a generic compression scheme. We found that compression rates improve over previous proposals by 161%, 124% and 83% for UI, 2D and 3D applications, respectively.

Figures

Tables

Table 1. Table 1: Baseline Configurations

Baseline DCP Configurations
FVC Size	64 entries
FVC replacement policy	Least-frequent value
CCD size	64 entries
Pixel Block size	8$\times$ 8
Pixel Sub-block size	2$\times$2
CSB bits per sub-block	1 (DCP, ADCP & HuffDCP)
	3 (VDCP), 5 (HDCP)
Memory Burst Size	128 bits
Pixel Sampling Rate	1:1

Table 2. Table 2 : List of Android workloads

#	Cat.	Benchmark	#	Cat.	Benchmark	#	Cat.	Benchmark	#	Cat.	Benchmark	#	Cat.	Benchmark
1	UI	Android Settings	26	UI	Pocket	51	UI	Yellowpages	76	UI	Textra	101	2D	Unwind
2	UI	Morecast	27	UI	ES File Explorer	52	UI	Eye in the Sky	77	UI	WPS Office	102	2D	Color Switch
3	UI	Poweramp	28	UI	Chrome	53	UI	OfficeSuite	78	UI	People Contacts	103	2D	Impossible Game
4	UI	Speedest	29	UI	Applock	54	UI	Dictionary.com	79	UI	Unit Conv. Ult.	104	2D	Flow
5	UI	Twitter	30	UI	Accuweather	55	UI	Walgreens	80	UI	Skyscanner	105	2D	2048
6	UI	Facebook	31	UI	Flipboard	56	UI	Walmart	81	UI	Calendar	106	2D	Gyro
7	UI	Twitch	32	UI	Booking.com	57	UI	CNN	82	UI	Merriam Webster	107	2D	99 Problems
8	UI	Wish	33	UI	Shazam	58	UI	File Commander	83	UI	ESPN	108	2D	Dumb Ways to Die
9	UI	Imgur	34	UI	Zedge	59	UI	Terminal Emulator	84	UI	Tumblr	109	2D	Piano Tiles
10	UI	Soundcloud	35	UI	Indeed	60	UI	Adobe Acrobat	85	UI	Quickpic	110	2D	loop
11	UI	Automate	36	UI	Runkeeper	61	UI	Android Call	86	UI	Duolingo	111	2D	Ultraflow
12	UI	Musixmatch	37	UI	Steam	62	UI	Gallery	87	UI	Clock	112	2D	Okay
13	UI	Airbnb	38	UI	Khan Academy	63	UI	Feedly	88	UI	Google Messenger	113	3D	Traffic Rider
14	UI	CBS Sports	39	UI	The Weather Channel	64	UI	Baconreader	89	UI	Calculator	114	3D	Extreme Car Driving
15	UI	Etsy	40	UI	Yahoo Finance	65	UI	aCalendar	90	UI	Soundhound	115	3D	3D Bowling
16	UI	Android Home	41	UI	Tapatalk	66	UI	Bakareader	91	UI	Translate	116	3D	Dr. Driving
17	UI	Pinterest	42	UI	Kickstarter	67	UI	Kindle	92	UI	Any.do	117	3D	Paper Toss
18	UI	Aldiko	43	UI	Amazon Store	68	UI	eBay	93	2D	Candy Crush Saga	118	3D	Rolling Sky
19	UI	Letgo	44	UI	Zomato	69	UI	Venmo	94	2D	Trainyard	119	3D	Stack
20	UI	Yelp	45	UI	Spotify	70	UI	Mcdonalds	95	2D	Mines	120	3D	Zigzag
21	UI	Android Messaging	46	UI	Runtastic	71	UI	Colornote	96	2D	Cut the Rope 2	121	3D	Stargather
22	UI	BBC iPlayer	47	UI	theScore	72	UI	Reddit	97	2D	Angry Birds	122	3D	Commute H. Traffic
23	UI	Tachiyomi	48	UI	Food Network	73	UI	Checkout 51	98	2D	Strata	123	3D	Crossy Road
24	UI	gReader	49	UI	MX Player	74	UI	Tasker	99	2D	Brain it On	124	3D	Smashy Road
25	UI	Google Maps	50	UI	VLC	75	UI	IFTTT	100	2D	Super Hexagon

Table 3. Table 3: System configurations and workloads summary

System Configurations
Operating System	Android 4.2.2 (API 17)
Display Size	720$\times$1280
Android workloads
Category	# of workloads
UI Applications	92 (24031 frames)
2D Applications	20 (7888 frames)
3D Applications	12 (2549 frames)
Total # of Applications	124 (34468 frames)

Table 4. Table 4: DCP structures hardware cost.

Structure	Type	Area cost (mm2)	Leakage power (mW)	Access cost (pJ)	Access latency (ns)	Max bandwidth (MPixels/s)
FVC	CAM	0.00304232	0.881899	0.766572	0.131695	7241
CCD	CAM	0.00197988	0.39	0.402	0.128338	7431
rCCD	Cache	0.000281592	0.281112	0.104106	0.0722227	13203

Equations

$$\text{FVC coverage}=\frac{\text{Sum of FVC frequencies}}{\text{Number of samples}}$$

Full text

In this work, we study the compression properties of framebuffer surfaces and highlight the fact that surfaces from different applications have different compression characteristics. We use the results of our analysis to propose a scheme, Dynamic Color Palettes (DCP), which achieves higher compression rates with UI and 2D surfaces.

DCP is a hardware mechanism for exploiting inter-frame coherence in lossless surface compression; it implements a scheme that dynamically constructs color palettes, which are then used to efficiently compress framebuffer surfaces. To evaluate DCP, we created an extensive set of OpenGL workload traces from 124 Android applications. We found that DCP improves compression rates by 91% for UI and 20% for 2D applications compared to previous proposals [1, 2]. We also evaluate a hybrid scheme that combines DCP with a generic compression scheme [1], and found that compression rates improve over previous proposals [1, 2] by 161%, 124% and 83% for UI, 2D and 3D applications, respectively.

1 Introduction

Off-chip memory traffic, including that of framebuffer surfaces, is one of the major sources of power consumption on mobile systems-on-chip (SoCs). In some cases, the energy consumption to access data on the off-chip memory can dominate that from computations [3]. In this work, we study the properties of framebuffer surfaces and propose a set of unique compression techniques to reduce the bandwidth consumed by framebuffer operations.

In graphics rendering, a framebuffer surface is an off-chip memory space that contains pixels generated by the graphics processing unit (GPU) and then used by the display controller to read pixels to the screen. In some cases, the display controller operates on multiple framebuffer surfaces, which are composited to a single surface for screen display. Also GPUs can use framebuffer surfaces as inputs to additional rendering stages, e.g., render to texture and deferred shading; as a result, any given application may utilize one or more framebuffer surfaces.

This work studies a large set of Android workloads to infer the compression properties of framebuffer surfaces generated by mobile UI, 2D and 3D applications. Our study found that framebuffers from different classes of workloads have different compression properties. We exploit these properties to propose an effective palette-based framebuffer compression scheme that focuses on common UI and 2D applications. In addition, we exploit temporal coherence in graphics, where applications exhibit minor changes between frames that can be exploited for compression.

Using temporal coherence, and by focusing on common uses cases, we propose and evaluate our Dynamic Color Palettes (DCP) technique. DCP uses palette based compression and focuses on reducing the traffic caused by framebuffer operations in UI and 2D applications. To evaluate our compression scheme, we created an extensive set of workloads from 124 Android applications. We show that by combining DCP with other compression techniques [1], DCP is able to improve compression rates between 83% and 161% across UI, 2D and 3D applications.

This paper makes the following contributions:

Characterizes compression properties of framebuffer surfaces from user-interface (UI) as well as non-UI 2D and 3D applications; 2. 2.

Uses characterization results to propose and evaluate dynamic color palettes (DCP), a compression technique that offers higher compression rates for common UI and non-UI 2D applications; 3. 3.

Proposes two DCP variations that dynamically choose an optimal palette size based on the frequencies of the values in color palettes; 4. 4.

Evaluate our compression schemes using an extensive set of workloads created from the OpenGL traces of 124 Android applications.

2 Background and Related Work

2.1 The life-cycle of a framebuffer surface

Figure 1 summarizes the life-cycle of a frame surface in contemporary mobile systems (Android Ice Cream Sandwich 4.0 and later [4]).

Figure 1 shows a typical scenario of drawing multiple surfaces simultaneously from multiple processes: the status bar, Facebook, and the navigation bar. Each process independently renders to its own surface ( 1 ); for example, Facebook renders a new surface when the user scrolls or clicks, while the navigation bar updates the corresponding surface when the user clicks on one of its buttons.

For display, a system compositor, such as SurfaceFlinger [4] in Android, combines surfaces from multiple applications before sending them to the screen ( 2 ). The compositor actively monitors the surfaces of all applications and when a process updates a surface, the compositor subsequently updates the composited surface. Simultaneously, the display controller hardware continuously reads the composited surface to the screen at 60 frames per second (FPS) or higher ( 3 ). Note that because using the same surface for updates and screen refresh operations can cause artifacts, such as flickering and tearing, double (or triple) buffering is used [5].

The example in Figure 1 shows how a surface can be used and re-used multiple times and this is why it is important to reduce the overhead of framebuffer manipulation through compression.

2.2 Surface compression techniques

Surface compression is used to reduce off-chip memory traffic, which can improve performance and/or reduce energy consumption. Graphics pipeline implementations utilize compression for textures [6], surfaces [1, 7, 8], depth [9] and vertex data [10].

Many of the compression techniques use lossy compression as well on lossless compression. For framebuffer surfaces, lossless compression is used to avoid error accumulation upon reading then re-writing surfaces (as is the case with composition).

Surface compression differs from texture compression in that both encoding, as well as decoding, are performed in real-time. Also opposite to surface compression schemes, most texture compression algorithms are lossy [6, 11, 12].

Another crucial aspect of surface compression is random accessibility. Techniques like Run-Length Encoding (RLE) are unable to provide such accessibility. However, it important to be able to randomly access a surface when used for sampling (e.g., used as a texture), resizing, or composition. Compression algorithms have used block-based schemes to enable random access for their simplicity and practicality. Block-based compression mechanisms define preconfigured compression sizes that allow random access to compressed surfaces. Block-based mechanisms have been used for compressing integer (e.g., RGBA) surfaces [1], floating-point surfaces [13, 14] and depth buffers [9, 14].

The work by Rasmusson et al. [1] (which we refer to by RAS) evaluated several surface compression proposals [15, 16, 17] and compared them against their technique. RAS is a lossless block-based compression technique for integer buffers that encodes the difference between adjacent pixel values. RGB pixels are converted to the $Yc_{o}c_{g}$ (luminance-chrominance) format, to increase compression efficiency. We compare against RAS in this paper since it reports better compression results versus prior work.

We also evaluate our scheme against the compression scheme proposed by Nvidia [2]. In this scheme, for each block going to memory, the algorithm checks if 4$\times$2 pixels in sub-blocks within a block are identical. If so, the block is compressed 1:8. When that is not possible, the algorithm then checks if 2$\times$2 regions have identical colors, if so the block is compressed 1:4, otherwise the block remains uncompressed. This algorithm works well with regions of identical color values, as the case with UI surfaces.

Other compression work includes the work by Danielson [18], which proposes using a dictionary-based compression in which the operating system and/or program specify the colors to configure a dictionary. In contrast to Danielson’s work, our work exploits temporal coherence to dynamically construct dictionaries (palettes) avoiding the need for software changes.

Another work by Shim et al. [19] use a dictionary-based compression mechanism targeted at display buffer compression. Shim et al.’s approach compresses surfaces using Huffman coding after rendering is completed to reduce the bandwidth of display refresh operations. Rendered surfaces are read to construct critical color differences which are used in a second stage to construct a Huffman dictionary. The third stage re-reads the surface buffer and writes out a compressed buffer that is then used for screen refresh operations. In contrast to previous work, we propose employing temporal coherence to predict the values for the dictionary, avoiding submitting uncompressed surfaces to memory or requiring additional surface read/write operations. Also we propose an adaptive compression scheme that avoids Huffman coding inefficiencies with probability distributions that are not exact powers of two.

Finally, a body of work has exploited temporal coherence in real-time rendering through inter-frame data reuse. These techniques, in addition to off-line rendering techniques like ray-tracing, are summarized in the survey work of Scherzer et al [20]. Here we propose a different application for temporal coherence by exploiting it for compression.

2.3 GPU Architectures

GPU architectures are broadly categorized as either tile-based or immediate-mode architectures.

Tile-Based architectures aim to save bandwidth by handling all raster operations, like blending and depth testing, using an on-chip buffer. Most mobile GPUs are tile-based architectures (including Qualcomm, Imagination and ARM GPUs).

In tiled rendering, the screen is divided into render tiles (e.g., 32$\times$32 or 64$\times$64 pixels). For each tile, the GPU renders all primitives that map to that tile using an on-chip buffer before committing that buffer to the off-chip memory. As a result, what is being compressed and sent to the off-chip memory is the final surface value of each tile.

On the other hand, immediate-mode architectures render primitives in their drawing order. They avoid the overhead of the tiling process, but the values sent to the off-chip memory will contain intermediate surface values. In the case of overdrawing (i.e., when a pixel location is covered by more than one primitive), the same memory location will be written to multiple times. When a compression scheme is deployed in an immediate-mode GPU, it will compress the values sent to the off-chip memory as rendering progresses; this means compressing blocks from the GPU’s LLC instead of an on-chip buffer.

As tile-based architectures are the dominant choice for mobile GPUs, going forward, we assume a tile-based architecture when evaluating surfaces for compression.

2.4 Mobile Use Patterns

In this work, we focus on developing an effective scheme for compressing UI and 2D framebuffer surfaces. The reason for this choice is that studies found users to spend 70% of their time running UI applications [21, 22], where over half of the time is spent on web browsing, messaging and social media alone. Whereas, games of all types, 2D, and 3D, account for only 30% of the usage time. Thus, we saw the opportunity of designing an effective scheme that targets such common use cases. In Section 7.5, we show how our DCP scheme can be combined with other generic compression algorithms to provide a comprehensive compression solution for all use-cases.

3 Temporal Coherence in Mobile Graphics

Temporal coherence is the property of inter-frame similarity [20]; this means that in a sequence of frames, content only gradually changes from one frame to the next. To quantify temporal coherence, we use two measurements: Color change and Pixel change.

Color change is the total difference in pixel color frequencies between two frames regardless of the locations of the pixels. On the other hand, Pixel change is the total number of pixels that change color between frames, which is measured by counting the number of pixel locations that differ in value between two frames. Color change estimates how similar two frames are, only with regard to color frequency. While Pixel change captures the movement of content on a surface.

To illustrate color and pixel change, we use an example from the Google Chrome browser in Figure 2. The example shows pixel and color change for different events. Notice that in some cases, when a new content is displayed, both pixel and color change values are high (e.g., new web search). In most cases, however, pixel change is always higher than color change; this means that in many cases the content is moving but not changing, as the case with BBC news in Figure 2.

Looking at a range of mobile workloads, we found that temporal color coherence is reflected by low Color change values, especially in UI applications. We analyzed a set of nine Android UI applications and games (UI: Twitter; Facebook; Chrome; and Android Home Screen, and 3D: Fruit Ninja; Need 4 Speed; Gunship 2; and Temple Run 2). In 3D applications, color and pixel change rates are 15.7% and 65%, respectively. On the other hand, UI applications has rates of 3.3% and 14.5% for color and pixel change values, respectively. These numbers show that 2D and UI applications exhibit higher temporal color coherence relative to 3D applications.

In addition to higher temporal coherence, we found that UI applications tend to use fewer colors. Figure 3 demonstrates how a small number of frequent pixel color values dominates a typical UI application compared to a 3D one. Figure 3 shows the cumulative distribution function of colors used in Twitter UI (a), compared to a 3D game, Temple Run 2 (b). In (a), the top 100 most common color values cover over 80% of the frame’s surface, while coverage is 10% for Temple Run 2. Measuring compressibility with Shannon entropy, we found that Twitter has an entropy of 4.5 bits per pixel, while it is 14 bits per pixel for Temple Run, indicating higher compressibility for Twitter.

The next section shows how to take advantage of temporal coherence and the color characteristics of UI applications to design a dynamic color palette scheme for compressing UI and 2D surfaces.

4 Dynamic Color Palettes (DCP) Compression

DCP is a technique to exploit graphics temporal color coherence for framebuffer compression. For each frame, DCP carries two operations in parallel: color frequency collection and framebuffer compression. For color frequency collection, DCP tracks the most frequently used colors as the rendering of a frame progresses; meanwhile, DCP works on compressing the pixels in the frame with a palette constructed using the frequency information of the previous frame.

DCP has two main advantages over previous dictionary-based techniques [18, 19]. First, it employs sampling to exploit temporal coherence to predict future dictionary values on-the-fly, alleviating the need for software hints or a multi-stage dictionary update process. This allows DCP to compress intermediate surfaces (i.e., application surfaces) as well as the framebuffer surface used by the display unit. Second, as we will show later, DCP maximizes compression using adaptive dictionary sizing, which puts to use the color frequency data collected for each frame.

DCP relies on two structures (shown in Figure 4), the Frequent Values Collector (FVC) for color frequency collection, and the Common Colors Dictionary (CCD) for compressing new pixels. The FVC identifies most commonly occurring colors, while the CCD encodes the most frequent colors as identified by the FVC from the previous frame. As shown in Figure 4, each frame the FVC collects color frequency information that are then used to construct the CCD of the next frame.

4.1 DCP workflow

Figure 5 shows DCP workflow. In
1 , the GPU commits tiles to the off-chip memory in multiple batches, i.e., blocks of spatially adjacent pixels [23, 24, 25]. For the example in Figure 5, we use a block size of 4$\times$4 pixels and a sub-block size of 2$\times$2. Pixels in each block are sent to the FVC  a1 and the CCD  b1 .

In
a1 , the FVC uses pixel values in each block to update the common color frequencies of the current frame (more details on that in the next Section).

In
b1 , the CCD compresses pixel blocks in batches of sub-blocks. In  b2 , if all pixel values in a sub-block have an entry in the CCD, the sub-block is determined to be compressible. Each color value in a compressible sub-block is represented using $log_{2}($CCD size$)$ bits, e.g., 6 bits per pixel for a CCD with 64 entries. If one of the pixel value in a sub-block does not have a CCD entry, then the whole sub-block remains uncompressed. Compressed and non-compressed sub-blocks are buffered and once a full block is processed, it is then written to the off-chip memory  b3 .

Like other block-based compression schemes [1, 26], DCP uses an a metadata compression status buffer (CSB) that contains a compression status bit for each sub-block. Upon compressing a sub-block, the corresponding entry in the CSB is set  b4 , and upon reading a compressed surface, the CSB is consulted to determine how much data should be fetched from memory.

4.2 The Frequent Values Collector (FVC)

FVC is a relatively small–e.g., 16 to 128 entries–associative memory structure. The FVC stores a set of pixel values and their corresponding frequencies as value-frequency pairs. For each pixel access, the FVC determines if a pixel already has an entry in the FVC, if so, the FVC increases the corresponding frequency counter by one. However, because FVC size is limited, the FVC uses an eviction policy to determine which pixel frequencies to keep track of. Similar to a fully associative cache, the FVC uses the least frequent color (LRC) policy, where it evicts the pixel value with the smallest frequency when an entry is needed to track the frequency of a new pixel value.

Hardware Cost

Each FVC entry contains a color value (32 bits for RGBA), a validity flag (1 bit), and a counter with $log2$(number of screen pixels) bits. For example, a 64-entry FVC sized for a 4k$\times$4k display will only require 456 bytes of storage.

4.3 The Common Colors Dictionary (CCD)

CCD is used to encode compressed pixels. At the end of each frame, the FVC holds the frequencies of the estimated most common colors. The FVC is then used to construct the CCD for the next frame. Each CCD entry maps a pixel value to a dictionary (encoding) value. The CCD is implemented using a fully associative structure.

When reading a surface, the mapping of CCD is reversed to decompress encoded pixels. We call the direct mapped structure that holds this reversed mapping the rCCD. Upon compressing a frame, or a set of frames, the rCCD mapping is attached to the frame and stored in main memory. Later on, when the frame is read, the rCCD is used to decompress the frame as described in Section 4.5 below.

Hardware Cost

CCD/rCCD with 64 entries only requires 264 bytes of storage.

4.4 The Compression Status Buffer (CSB)

Similar to other block-based compression algorithms [1, 13, 14, 9], a metadata buffer is used to hold the status of each compression block. For DCP, the CSB buffer indicates whether a given sub-block is compressed, where CSB holds one bit per sub-block. In our baseline, this translates to a cost of 1 bit per 128 bit of surface data. Both CSB and rCCD are needed to read a compressed frame as explained in the next section.

4.5 Reading a Compressed Framebuffer Surface

Figure 6 shows the process of reading a compressed surface. It starts with loading the corresponding rCCD and CSB. To read a pixel, CSB entries are decoded to determine the size of compressed data and how many bytes should be fetched for each block. To avoid double latency, and since CSB size is relatively small, the CSB can be prefetched to a small on-chip buffer/cache. Once CSB is used to determine the size of a compressed block, the block is then fetched and the rCCD is used to decompress the values in each sub-block as shown in Figure 6.

4.6 Multi-Surface Support

Multiple Render Targets (MRT): Some graphics applications may render to multiple target surfaces. Techniques that use MRT, like deferred shading, are popular in 3D applications and used to render scenes with complex lighting [27]. To support multiple render targets, we need to replicate some of the structures in Figure 5 to match the maximum possible number of target surfaces. DCP will need a single FVC and a single CCD unit per render target. However, no need for additional FVC and CCD units if multiple passes are used to process MRT.

Since most UI and 2D workloads render to a single target, a typical hardware implementation may only need support a single render target and the rare case of multiple targets is handled by using DCP with just a single surface. However, as discussed in Section 7.4, adding extra structure is relatively cheap and cost little chip area.

Multi-Surface Composition: Contemporary compositor engines can composite up to 16 surfaces in one pass [28]. To support multi-surface composition, the number of rCCD structures in Figure 6 should match the number of surfaces that can be composited in parallel.

4.7 Coupling DCP with Other Compression Algorithms

DCP targets common UI and 2D applications. Other compression algorithms are better suited to 3D and some 2D applications. Industry practitioners have proposed supporting multiple compression algorithms [2, 26]. This means that in a hybrid scheme, each block can be compressed either using DCP or an alternative algorithm. In Section 7.5 we evaluate the results of combining DCP with RAS.

4.8 Dynamically Enabling DCP

In this section, we explain how DCP can be enabled/disabled based on the expected compression performance. DCP performance can be predicted using the frequencies collected by the FVC at the end of a frame. By adding frequency values in the FVC, then comparing it to the total number of pixels (sample size), we can calculate what we call FVC coverage, which can be used to predict DCP performance, where:

[TABLE]

By defining a coverage threshold (CT) and comparing it to FVC coverage, then DCP can be used only if FVC coverage $\geq$ CT. By periodically enabling FVC, e.g., once every $n$ frames, FVC coverage can be updated and used to determine if DCP should be enabled. For an N-entry FVC, calculating coverage takes $N-1$ integer addition and one division operations per frame.

Figure 7 shows FVC coverage vs. compression rates across workloads in Table 3. It is clear that higher compression rates are achieved with higher FVC coverage. In our set of workloads, using DCP with FVC coverage $\geq$ 0.7 seems to achieve good compression rates ($>$ 2).

Figure 7 also shows some cases where larger FVC coverage yields lower compression. These cases represent workloads that exhibit sudden changes in frames, as a result, temporal coherence is lower than that of other benchmarks with similar FVC coverage. Two examples from Figure 7 (the two large dips at the right end) are Unwind which exhibits a UI with changing color brightness and Super Hexagon which exhibits an interface that continuously switches theme colors.

5 DCP Schemes

5.1 Baseline DCP

In baseline DCP, the CCD is constructed using all FVC entries; thus, the number of entries in the CCD will always match FVC, and compressed blocks will have a fixed size of $log2$(FVC size)$\times$ (pixels per block) bits.

Memory layout and effective compression rates

Figure 8 shows the memory layout of a DCP compressed surface. Space allocated to DCP blocks (0-2) is fixed ($S_{0}$, i.e., the size of an uncompressed block). On the other hand, the actual utilized space is determined by the size of compressed data ($S_{2}$). But because DRAM reads/writes data blocks using a number of bandwidth cycles that are burst size multiples, a block that should be compressed by $S_{0}/S_{2}$ will have an effective compression rate of $S_{0}/S_{1}$, where $S_{1}$ is the size of DRAM bursts needed to read compressed data.

In this work, we use the effective compression rate which reflects the reduction in memory bandwidth. In the remainder of this section, two variants of DCP (ADCP and VDCP) are introduced in addition to a hybrid scheme combining DCP and RAS (HDCP).

5.2 Adaptive DCP (ADCP)

ADCP is a variation of DCP that uses the distribution of frequent color values in the FVC to adjust the number of CCD entries. ADCP looks for the best trade-off between the number of compressible blocks and the size of their encoding.

Frames of different applications, or different frames within the same application, may perform better/worse under larger/smaller palette sizes. Figure 9 shows DCP compression rates for Facebook and Kindle using 16 to 512 entry CCDs. Kindle with its simple text achieves higher compression rates using smaller CCDs. On the other hand, Facebook achieves the best compression rate using a 256-entry CCD. A larger CCD covers a wider range of values and it is able to compress more blocks, while a smaller CCD uses smaller encoding sizes. For example, if a frame that uses 32-bit pixels with blocks that are 80% white, 18% blue, 1% black and 1% red uses a 2-entry CCD, 98% of the blocks can be compressed using 1 bit per pixel for a total compression rate of 19.75$:$1 (ignoring metadata overhead). Another option is to use a 4-entry CCD to compress all the frame using 2 bits per pixel producing a compression rate of 16$:$1.

ADCP tries to optimize CCD size for each case by actively predicting the optimal number of CCD entries. CCD size determines encoding sizes and subsequently the size of compressed blocks. ADCP uses FVC to predict the optimal CCD size using Algorithm 1. In Algorithm 1, FVC frequencies, sorted from most to least frequent in FVC_Val, are used as input. Note that to simplify calculations, DRAM burst size and pixels layout were ignored.

ADCP has a negligible overhead; the number of iterations in Algorithm 1 depends on the number of FVC entries. For example, for 64-entry FVC, the loop will only execute six times (i.e., $log2$(FVC size)).

5.3 Variable DCP (VDCP)

VDCP is another DCP variation that goes further than ADCP by adapting palette sizes to optmize compression at the sub-block level. VDCP uses variable-length coding by changing the number of rCCD entries used to encode/decode each sub-block. VDCP reduces the number of encoding bits per pixel to $i=ceil(log_{2}(max(pixel\_color\_index)))$, which means that $i$ is determined by the pixel within the sub-block that has the highest index (i.e., lowest frequency) in the CCD.

With VDCP, CSB is used to determine the number of rCCD entries used for each sub-block, where the number rCCD entries equals to $2^{CSB\_Value}$ (i.e., encoded colors fall in the first $2^{CSB\_Value}$ CCD entries), and a special CSB_Value is used for uncompressed sub-blocks.

Figure 10 shows a VDCP example. In Figure 10.a, an example CCD is shown, where the most frequent color value, $C_{0}$, is encoded to 000 and the least frequent value, $C_{7}$, is encoded to 111. Figure 10.b shows the VDCP encoding for seven 2-pixel sub-blocks. As shown in the figure, the CSB tracks each sub-block’s encoding. 0002 in the CSB indicates that only the most frequent color in the CCD $C_{0}$ is used, while 0012 indicates that the top 2 CCD colors, {$C_{0}$ and $C_{1}$}, are used and so on. 1112 is used for uncompressed sub-blocks.

In Figure 10.b, the first row shows a sub-block with values C2 and C3, this means that only the top $2^{2}$ entries in the CCD are used for encoding the sub-block. Subsequently, the corresponding CSB entry is set to 0102, and each pixel color is encoded using 2 bits. The second sub-block in Figure10.b contains {C0,C1}, encoded using the top $2^{1}$ CCD entries (1 bit per color), and the corresponding CSB entry is set to 0012. The third sub-block contains only $C_{0}$, and the corresponding CSB value in this case is 0002, which indicates the content for the entire sub-block (since only the top entry in the table is used). Note that to make this example easy to follow, the CCD is shown with eight entries (instead of 64), so the CSB values 1002 to 1102 are not in use.

5.4 Hybrid DCP (HDCP)

DCP is only effective on a subset of applications. Ideally, it should be used with other compression algorithms. We evaluate a Hybrid DCP that combines DCP with RAS. HDCP compresses each block using DCP and RAS and uses the result with higher compression rate. To support the additional compression modes, the number of CSB bits is increased. Results in Section 6 show that this technique produces higher compression rates at the cost of additional on-chip computations.

5.5 DCP Implementation

In addition to hardware structures (FVC, CCD and rCCD), DCP requires some support from the software layer. To implement DCP, the graphics driver will attach DCP data as part of the state associated with a surface (along other state data like size and formatting). For VDCP, Algorithm 1 can be added to the driver as well, where it can calculate next CCD size at the end of each frame.

6 Methodology

Our experimentation configurations are listed in Table 1. We calculated compression rates using a model that assumes a tile-based GPU architecture. Our model works as follows:

• •

First, we feed the frames of each workload to our model, which then splits each frame to 8x8 blocks.

• •

For each block, the model calculates the compressed size of each sub-block. The total of compressed and uncompressed sub-block sizes are added to calculate the compressed size of the block.

• •

Compressed block size is then used to calculate the number of DRAM bursts required. The model then calculates the total bandwidth consumed by a compressed frame by summing the number of DRAM bursts of all the blocks in the frame.

Note that the model computes compression rates starting from the second frame, using the first frame to populate the first FVC and CCD.

We evaluated surface compression using our set of randomly chosen popular Android applications (Table 2). Our traces will be published and made available for any future studies.

We split applications into three groups: UI applications, 2D applications, and 3D games. All of our benchmarks use OpenGL ES and render to a single target buffer (up to OpenGL ES 2.0 MRT is only supported through vendor extensions [29]).

We manually interacted with each application to execute a simple task. In total, we used 34468 frames that represent 124 applications (shown in Table 3). We only consider regions of interest in each workload that represent the typical use case of the workload (i.e., loading/initialization frames are not considered).

The rest of configurations are listed in Table 1. The effective compression rate and metadata overhead are taken into account when calculating the total compression rate. We use a block size of 8$\times$8 pixels (256 bytes), which matches the block sizes used by RAS.

In addition to DCP, we evaluate two lossless methods described in Section 2. RED uses Nvidia’s compression [2] and RAS, which is based on work of Rasmusson et al. [1]. RAS is a prediction based algorithm that predicts the value of a pixel using neighbor pixel values. The difference between prediction and the actual value is then encoded using Golomb-Rice coding. We used parameters suggested by Rasmusson et al. [1], namely 8$\times$8 blocks and, as described in the paper, we set the value of the Golomb-Rice parameter k by exploring values between 0 and 6, use k = 7 for the “special mode”, and use the suggested “3 sizes mode” for higher compression rates. We organize color values by their color channel as described in Ström et al. [14]. We experimented with RAS using RGBA and $Yc_{o}c_{g}$ formats and found that for many applications, particularly UI and 2D, RAS shows favorable results using RGBA channels. So we used RAS with RGBA channels in our comparison.

For CSB, DCP and ADCP use 1-bit per sub-block. VDCP uses 3 bits per sub-block; with an FVC size of 64, seven combinations are used–1, 2, 4, 8, 16, 32 and 64, plus a combination for non-compressed sub-blocks. To compare against techniques that use Huffman coding [19], a Huffman coded DCP (HuffDCP) is implemented, where FVC frequencies are used to construct CCD with variable length Huffman coding.

7 Results and Discussion

7.1 DCP Schemes

To compare DCP schemes, we isolate the effect of memory burst size and only take into account CSB overhead. Later, bursts are taken into account when comparing DCP, RAS and RED. We compare baseline DCP against ADCP, VDCP and HuffDCP.

Figure 11(a) shows compression rates in each category sorted by baseline DCP compression rate (the same order used in Table 2). The figure shows that the baseline DCP is the least effective scheme. UI applications  1 , such as Android Settings, Morecast, and Poweramp, show low compression rates of less than 2. After examining these applications, we found that they feature gradient backgrounds and graphical elements that DCP cannot compress using small palettes.

In  2 (Zedge) and   3 (Spotify), ADCP achieves higher compression rates than HuffDCP and VDCP. Looking at these applications we found that they contain a mix of solid backgrounds and frames that contain images which DCP will, mostly, not be able to compress. ADCP can compress frames with solid backgrounds with lower overhead than VDCP since it has a lower CSB overhead. On the other hand, in frames containing images, both ADCP and VDCP will not be able to perform well, but ADCP will incur lower CSB overhead.

For applications with simple color schemes, such as OfficeSuite
4 and Any.do
5 , VDCP and DCP achieve high compression rates. Nevertheless, VDCP, ADCP, and HuffDCP were all able to achieve even higher compression rates. Looking at 2D applications, performance varies significantly.

In
6 , applications with sophisticated graphics, like Candy Crush, Trainyard, Mines, Cut the Rope and Angry Birds, have low compression rates ($<$ 1.7). On the other hand, applications using simpler graphics (e.g., loop Ultraflow, and Okay) achieve high compression rates, especially with VDCP
7 . A similar trend is exhibited in
8 , where graphically rich 3D games (Traffic Rider, Extreme Car Driving, and 3D Bowling) show low compression rates. On the other hand, games like Smashy Road, show good compression rates (highest VDCP at 10.63). Also Stargather
9 , with similar characteristics to UI applications in
2 and
3 , shows higher rates with HuffDCP and VDCP.

Figure 12 summarizes the results in Figure 11(a). VDCP shows better compression rates for UI and 2D applications with 5.56 and 3.02 respectively. For 3D games, HuffDCP shows the highest rate (1.80). HuffDCP and VDCP do better with 3D workloads since their compression rates are similar to VDCP but with lower CSB overhead. Interestingly, using Huffman encoding in HuffDCP achieves lower compression rates than VDCP in UI and 2D workloads. This is due to Huffman inefficiencies with probability distributions that are not exact powers of two. For example, if we have 32 bit values and frequencies of A(49.5%), B(49.5%), C(0.5%) and D(0.5%) then Huffman encoding will assign codes of 1 bit to A, 2 bits to B and 3 bits to C and D with a total compression rate of 21.12. ADCP and VDCP encode A and B using 1 bit, while keeping C and D uncompressed, resulting in a compression rate of 24.4.

7.2 Comparing VDCP, RAS and RED

Figure 11(b) compares VDCP with RAS and RED and Figure 13 summarizes the results in Figure 11(b). Memory bursts and CSB overhead were taken into account. For UI applications, VDCP achieves a mean effective compression rate of 5.26 compared to 2.73 for RAS and 2.77 for RED. VDCP performs well with UI and 2D applications. On the other hand, RAS, a more generic compression algorithm, has consistent performance across all workloads. RAS outperforms VDCP in 3D games (2.54, compared to 1.75 for VDCP). Similar to VDCP, RED performs well with UI and 2D applications, but with lower rates that VDCP. VDCP performance with 3D workloads is the reasoning behind suggesting a hybrid approach consisting of DCP and another general purpose compression algorithms–similar to what is described some implementations [2, 26]. A Hybrid VDCP-RAS scheme is discussed in Section 7.5.

7.3 Factors affecting FVC Fidelity

In this section we discuss and quantitatively evaluate four factors that affect FVC and should be considered when using DCP.

FVC Size:

Larger FVC sizes can capture frequent colors more accurately, as they are less likely to evict a frequent value from the FVC because of capacity.

To evaluate how FVC size affect accuracy, we use relative coverage. For an $N$-entry FVC, we calculate relative coverage by dividing the number of pixels represented by the $N$ top colors collected by FVC by the number of pixels represented by the actual $N$ most frequent colors.

Figure 14(a) shows the effect of FVC size for UI applications. A 16-entry FVC has a relative coverage of 94% compared to 98.3% for 512-entry FVC. This means a 16-entry FVC is able to capture colors that cover 94% of the area covered by the actual 16 most frequent colors, while the 512-entry FVC is able to capture 98% of the coverage the actual 512 most common colors are able to cover. Figure 14(b) shows how accuracy affect compression rates, as frequencies collected using larger FVCs are a better representation of the actual most common colors.

Replacement Policy and Associativity

We evaluated using a number of replacement policies: the baseline least-frequent color (LFC), second least-frequent (2LFC), least-recently-used (LRU), and random replacement. The idea behind including a 2LFC is to see the effect of avoiding thrashing newly discovered colors that are prone to eviction.

Using UI workloads with 64-entry FVC, the mean compression rate with LFC is 5.26, while it is 5.25 for 2LFC. On the other hand, LRU and random achieve lower rates of 3.04 and 2.92, respectively.

We also evaluated changing FVC associativity from fully associative to direct-mapped, and used color channel values to determine the set. As expected, the FVC performance degrades as we increase the number of sets (as shown Figure 15).

Pixel Sampling

We noticed that the FVC can be constructed using a subset of frame pixels, i.e., by sampling them using only one in every $n$th pixel to collect frequent colors statistics.

Figure 16 illustrates the effect of pixel sampling on VDCP. We evaluate sampling rates from 1:1 (every pixel accesses the FVC) to 1:16384. 1:16 sampling achieves 98.7% (UI) and 102% (2D) of the compression achieved by 1:1 sampling. We expect that the slightly higher compression rate for 2D workloads is caused by sampling working as a noise filter.

Frame sampling

In frame sampling, the same CCD is used for a number of frames ($N$) instead for just one frame. We vary the sampling period ($N$) for VDCP between 1 (every frame) and 60 frames. Figure 17 shows compression rates relative to $N$=1. VDCP maintains good compression rates with $N$=2, with a relative compression rate of 97%. Compression rates, however, significantly decrease with higher $N$ values with 44.6% and 43.3% for $N$ values of 50 and 60, respectively.

7.4 Implementation Cost and Energy Savings

Section 5 mentions storage requirements associated with DCP. Specifically, for 64-entry FVC/CCD, 456 bytes are need for FVC and 264 bytes for CCD. The cost of rCCDs is (264 bytes) x (maximum number of surfaces that can be read in parallel). Current systems support up to 16 surfaces [28].

For energy, we used DRAMPower v4.0 [30] to estimate the energy cost of accessing a MICRON 1600_x32 LPDDR3 DRAM. We found the cost of DRAM accesses to be around 451.2 pJ/byte (this number excludes DRAM idle energy and other system energy costs like the interconnection network). For DCP we used CACTI v7.0 [31] with the 22nm process to estimate the area/energy/latency of DCP structures as shown in Table 4.

Using the numbers in Table 4, DCP total area cost with support of 16 surfaces equals to 0.009527672 mm2. To compare this area with current hardware, it is less than 0.003% of Nvidia’s Xavier die area [32]. For the the dynamic energy cost of compressing/decompression a byte using DCP, we found it to be around 1.3 pJ/byte, i.e., less than 0.29% of DRAM access cost.

Energy savings

DRAM consumes around 199.6 mW (629.4 mW including static power) for framebuffer operations under a typical rate of 60 FPS using HD frames (GPU writing/display controller reading, or 949.21 MB/s). We calculated DCP total compression/decompression static and dynamic energy consumption (4.83 pJ/byte) and we compared it to only DRAM dynamic energy consumption (451.2 pJ/byte). We found that VDCP reduces the energy consumed by framebuffer operations by 79.9% for UI apps, 64.4% for 2D apps, and 41.8% for 3D apps.

7.5 Hybrid Schemes

Our hybrid compression scheme uses RAS and VDCP. We compress using both algorithms and then use the best of the two. This exploits VDCP high compression rates for simpler surfaces while falling back on RAS for other cases. RAS+VDCP outperforms RAS and VDCP (with rates of 7.2, 5.206 and 3.23 for UI, 2D and 3D applications respectively). The ratio of VDCP vs. RAS compressed blocks varies by application as shown in Figure 18. However, we found that, on average, VDCP and RAS compress an equal number of blocks.

8 Conclusion

This work presents surface compression techniques that reduce the off-chip bandwidth of framebuffer operations in energy-constrained mobile devices. In this work, we analyze and characterize the framebuffer surfaces of UI, 2D and 3D applications and highlight the unique characteristics of each.

To evaluate our compression schemes, we created and used a set of workloads that represents 124 popular mobile applications. Our results show that VDCP improves compression by an average of 93% relative to RAS for UI applications, while improving UI and 2D applications over RED by 89% and 50%, respectively.

DCP focuses on 2D and UI applications and can complement other generic compression algorithms. We evaluated a hybrid VDCP+RAS (HDCP) scheme; the scheme was able to increase compression rates by 163%, 79% and 27% over RAS, and by 159%, 169% and 139% over RED for UI, 2D and 3D applications, respectively.