JPEG XT Image Compression with Hue Compensation for Two-Layer HDR Coding

TL;DR

This paper introduces a hue compensation technique for JPEG XT HDR image compression that reduces color distortion while maintaining standard compatibility, improving hue accuracy and luminance mapping.

Contribution

It presents a novel hue compensation method based on maximally saturated colors that enhances color fidelity in JPEG XT HDR images without standard incompatibility.

Findings

Reduces hue distortion in LDR images from JPEG XT bitstreams

Maintains luminance mapping quality across different criteria

Ensures full compatibility with JPEG XT standard

Abstract

We propose a novel JPEG XT image compression with hue compensation for two-layer HDR coding. LDR images produced from JPEG XT bitstreams have some distortion in hue due to tone mapping operations. In order to suppress the color distortion, we apply a novel hue compensation method based on the maximally saturated colors. Moreover, the bitstreams generated by using the proposed method are fully compatible with the JPEG XT standard. In an experiment, the proposed method is demonstrated not only to produce images with small hue degradation but also to maintain well-mapped luminance, in terms of three kinds of criterion: TMQI, hue value in CIEDE2000, and the maximally saturated color on the constant-hue plane.

Figures

Equations

$$\mathbf{x}=a_w\mathbf{w}+a_k\mathbf{k}+a_c\mathbf{c},$$

$$\left\{\begin{array}[]{l}a_{w}=\min(\bm{x})\\ a_{k}=\max(\bm{x})-\min(\bm{x})\\ a_{c}=1-\max(\bm{x})\\ \end{array}\right.,$$

$$a_w+a_k+a_c=1,$$

$$0\le a_w,a_k,a_c\le 1.$$

$$\left\{\begin{array}[]{l}c_{r}=(x_{r}-\min(\bm{x}))/(\max(\bm{x})-\min(\bm{x}))\\ c_{g}=(x_{g}-\min(\bm{x}))/(\max(\bm{x})-\min(\bm{x}))\\ c_{b}=(x_{b}-\min(\bm{x}))/(\max(\bm{x})-\min(\bm{x}))\end{array}\right..$$

$${\mathbf{x}}_H=a_{Hw}\mathbf{w}+a_{Hk}\mathbf{k}+a_{Hc}{\mathbf{c}}_H,$$

$$\left\{\begin{array}[]{l}c_{Hr}=(x_{Hr}-\min(\bm{x}_{H}))/(\max(\bm{x}_{H})-\min(\bm{x}_{H}))\\ c_{Hg}=(x_{Hg}-\min(\bm{x}_{H}))/(\max(\bm{x}_{H})-\min(\bm{x}_{H}))\\ c_{Hb}=(x_{Hb}-\min(\bm{x}_{H}))/(\max(\bm{x}_{H})-\min(\bm{x}_{H}))\end{array}\right..$$

$${\mathbf{x}}^{\prime }$$

$$=x_r^{\prime }\mathbf{r}+x_g^{\prime }\mathbf{g}+x_b^{\prime }\mathbf{b},$$

$$\left\{\begin{array}[]{l}x^{\prime}_{r}=a_{w}+a_{c}\cdot c_{Hr}\\ x^{\prime}_{g}=a_{w}+a_{c}\cdot c_{Hg}\\ x^{\prime}_{b}=a_{w}+a_{c}\cdot c_{Hb}\\ \end{array}\right..$$

Full text

I Introduction

The interest of high dynamic range (HDR) imaging has recently been increasing in various area: photography, medical imaging, computer graphics, on vehicle cameras, astronautics. HDR images have the information of the wide dynamic range of real scenes. However, commonly used display devices which can directly represent HDR images are not popular yet. Therefore, various two-layer encoding method to generate bitstreams with base layer and residual layer, have been proposed for compressing HDR images [1, 2, 3, 4, 5, 6, 7]. ISO/IEC JTC 1/SC 29/WG 1 (JPEG) has developed a series of international standards referred to as the JPEG XT[8, 9, 10, 11, 12]. The JPEG XT has been designed to be backward compatible with the legacy JPEG standard with two-layer coding.

For the base-layer coding, a low dynamic range (LDR) image is generated by a tone-mapping operation (TMO). In the operation, the range of the luminance of an HDR image is compressed by using a tone-mapping (TM) operator, and then an LDR image is produced by combining the compressed luminance and the color information of the original HDR image. However, TMOs only focusing on the luminance cause image colors to be distorted as pointed out in [13, 14, 15, 16, 17].

In this paper, we propose a novel JPEG XT image compression with hue compensation for two-Layer HDR coding. The proposed compensation method utilizes the hue information based on the maximally saturated colors[18], for suppressing color distortions due to the influence of TMOs.

To evaluate the effectiveness of the proposed method, we perform a number of simulations. In the simulations, the proposed method is compared with conventional TMOs in terms of the hue difference used in CIEDE2000[19], the maximally saturated color difference, and Tone Mapped image Quality Index (TMQI)[20].

II Proposed Method

II-A Overview

As shown in Fig. 1, a JPEG XT bitstream consists of base layer and residual layer. A tone-mapped LDR image is decoded from the base layer, and an HDR image is decoded by using both layers. However, the LDR image includes some hue distortion due to the influence of TMO. In order to suppress the hue distortion, we apply a hue compensation process based on the constant hue plane in the RGB color space before encoding LDR images. In the hue compensation process, a hue compensated image $\bm{X}^{\prime}_{L}$ is generated by using an LDR image $\bm{X}_{L}$ and the maximally saturated colors of the original HDR image. In addition, inverse TMO (TMO*′-1* in Fig. 1) is modified according to the hue compensation. These modifications are applied to only the JPEG XT encoder, and moreover generated bitstreams are fully compatible with the JPEG XT standard. As a result, LDR and HDR images are decoded from the bitstreams by using the standard JPEG XT decoder without any modification.

II-B Constant hue plane

We focus on the constant hue plane in the RGB color space[18]. An input image $X_{L}$ is a 24-bit full color LDR image and each pixel of the image is represented as $\bm{x}\in[0,1]^{3}$. $x_{r}$, $x_{g}$, and $x_{b}$ are the R, G, and B components of a pixel $\bm{x}$, respectively. In the RGB color space, a set of pixels which has the same hue forms a plane, called constant hue plane. The shape of the constant hue plane is the triangle whose vertices correspond to white ($\bm{w}=(1,1,1)$), black ($\bm{k}=(0,0,0)$) and the maximally saturated color ($\bm{c}$) with the same hue as $\bm{x}$. On the constant hue plane, a pixel $\bm{x}=x_{r}\bm{r}+x_{g}\bm{g}+x_{b}\bm{b}$ can be represented as a linear combination as

[TABLE]

where

[TABLE]

and $\max(\cdot)$ and $\min(\cdot)$ are functions that return the maximum and minimum elements of the pixel $\bm{x}$, respectively. Since $\bm{w}$, $\bm{k}$, $\bm{c}$ and $\bm{x}$ exist on the plane and $\bm{x}$ is an interior point of $\bm{w}$, $\bm{k}$ and $\bm{c}$, the following equations hold.

[TABLE]

Moreover, the maximally saturated color $\bm{c}=(c_{r},c_{g},c_{b})$ is given by

[TABLE]

From Eq. (5), the elements of $\bm{c}$ corresponding to the maximum and minimum elements of the pixel $\bm{x}$ become 1 and 0, respectively.

We consider the same pixel representation on the constant hue plane for HDR images and discuss the relation between HDR images and LDR ones. Each pixel of an HDR image $X_{H}$ is represented as $\bm{x}_{H}=(x_{Hr},x_{Hg},x_{Hb})$ where $x_{Hr}$, $x_{Hg}$, and $x_{Hb}$ are generally real numbers. As well as LDR images, we define white, black and the maximally saturated color as $\bm{w}=(1,1,1)$, $\bm{k}=(0,0,0)$ and $\bm{c}_{H}$, where $\bm{c}_{H}$ is calculated from Eq. (5) by replacing $\bm{x}$ with $\bm{x}_{H}$, and then the pixel value $\bm{x}_{H}$ is also represented as a linear combination as

[TABLE]

where $a_{Hw}$, $a_{Hk}$ and $a_{Hc}$ are coefficients calculated from $\bm{x}_{H}$, and the maximally saturated color $\bm{c}_{H}=(c_{Hr},c_{Hg},c_{Hb})$ is given by

[TABLE]

.

In general, $\bm{c}_{H}=\bm{c}$ is not satisfied, because some hue distortion occurs due to the influence of TMO. Therefore, to correct the hue distortion, we replace $\bm{x}$ with $\bm{x}^{\prime}$ as,

[TABLE]

where

[TABLE]

This process corresponds to ‘Hue Compensation’ in Fig. 1. Note that $\bm{x}^{\prime}$ and $\bm{x}_{H}$ are on the same constant hue plane.

II-C Procedure of hue compensation

The procedure of the proposed method shown in Fig. 1 is shown as follows.

Generate an LDR image $X_{L}$ by using a TMO. 2. 2.

Calculate $a_{w}$, $a_{k}$, and $a_{c}$ for each pixel value $x$ of $X_{L}$ by Eq. (2). 3. 3.

Calculate $\bm{c}_{H}$ for each pixel value $x_{H}$ of the original HDR image $X_{H}$ according to Eq. (7). 4. 4.

Generate a compensated image $X^{\prime}_{L}$ by Eq. (8). 5. 5.

Input $X^{\prime}_{L}$ to a JPEG XT encoder.

III Experimental results

III-A Hue distortion

We used five HDR images selected from an HDR image database[21]. We adapted two objective metrics to evaluate hue distortion caused in TMOs under the use of three TMOs: JPEG XT default TMO, Reinhard’s local operator[22], and Drago’s TMO[23]. One is $\Delta c$, which is the average of euclidean norms of the maximally saturated colors $\bm{c}_{H}$ and $\bm{c}$. The other is the hue differences defined in CIEDE2000 $\Delta H$, which was published by the CIE[19].

Table I shows evaluation results in terms of the objective metrics. In the table, the proposed method outperformed the conventional JPEG XT bitstreams for both TMOs. Therefore, the proposed method is demonstrated to be effective for improving hue distortion in terms of not only $\Delta c$ but also $\Delta H$.

III-B Influence on tone mapping

Next, we evaluate the quality of tone-mapped LDR images in terms of TMQI. From Tab. II, the proposed method is shown to offer almost the same TMQI scores as those of the conventional bitstreams. The results show that the proposed one allows us to compensate hue distortion, while maintaining TM performances.

III-C Visual evaluation

We used two HDR images, ‘McKeesPub’ and ‘WillyDesk’, to visually evaluate the quality of images. The images were encoded by the proposed encoder and the conventional JPEG XT one, respectively. In this experiment, the default TM operator of JPEG XT was used as a TM operator. Figure 3 shows decoded LDR images, where (a) and (b) were results produced by using the conventional approach, and (c) and (d) were generated by using the proposed one. From the result, we can confirm there are color differences in some places.

To more clearly demonstrate the differences, maximally saturated color images are shown in Fig. 4. The proposed method is demonstrated to have closer maximally saturated colors to those of the HDR images than the conventional method.

Figure 5 shows euclidean norms of the maximally saturated color $\bm{c}_{H}$ and $\bm{c}$. From these figures, the proposed method is also demonstrated to be effective in reducing hue distortion.

IV Conclusion

In this paper, we proposed a novel two-layer HDR coding method to reduce hue distortion of LDR images due to the influence of TMO. The proposed method enables us to compensate the hue distortion under the use of any TM operator, while maintaining well-mapped luminance. In addition, the proposed one has compatibility with JPEG XT bitstreams. Experimental results showed the effectiveness of the proposed method in terms of three objective metrics: $\Delta c$, $\Delta H$, and TMQI.