A hue-preserving tone mapping scheme based on constant-hue plane without gamut problem
Yuma Kinoshita, Kouki Seo, Artit Visavakitcharoen, Hitoshi Kiya

TL;DR
This paper introduces a hue-preserving tone mapping method that reduces color distortion in LDR images derived from HDR sources, applicable to any tone mapping operator, and maintains luminance quality.
Contribution
It presents a novel approach that preserves hue and maximally saturated colors in tone mapping, addressing hue degradation issues in conventional methods.
Findings
LDR images with smaller hue degradation compared to conventional TMOs.
The proposed method maintains luminance quality as measured by TMQI and CIEDE2000.
Applicable to any tone mapping operator, enhancing color fidelity in HDR to LDR conversion.
Abstract
We propose a novel hue-preserving tone mapping scheme. Various tone mapping operations have been studied so far, but there are very few works on color distortion caused in image tone mapping. First, LDR images produced from HDR ones by using conventional tone mapping operators (TMOs) are pointed out to have some distortion in hue values due to clipping and rounding quantization processing. Next,we propose a novel method which allows LDR images to have the same maximally saturated color values as those of HDR ones. Generated LDR images by the proposed method have smaller hue degradation than LDR ones generated by conventional TMOs. Moreover, the proposed method is applicable to any TMOs. In an experiment, the proposed method is demonstrated not only to produce images with small hue degradation but also to maintain well-mapped luminance, in terms of three objective metrics: TMQI, hue…
| Reinhard(global) | Reinhard(local) | Drago | Fattal | Shan | Gu | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Images | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. |
| Apartment_float | 0.0766 | 0.0672 | 0.1405 | 0.1318 | 0.0198 | 0.0133 | 0.0136 | 0.0099 | 0.0472 | 0.0400 | 0.0132 | 0.0089 |
| dani_belgium | 0.0130 | 0.0104 | 0.0388 | 0.0315 | 0.0150 | 0.0036 | 0.0103 | 0.0036 | 0.0274 | 0.0217 | 0.0131 | 0.0053 |
| Desk | 0.0227 | 0.0136 | 0.0655 | 0.0450 | 0.0276 | 0.0028 | 0.0166 | 0.0030 | 0.0439 | 0.0320 | 0.0238 | 0.0057 |
| Display1000_float | 0.0222 | 0.0184 | 0.0830 | 0.0737 | 0.0137 | 0.0050 | 0.0127 | 0.0076 | 0.0269 | 0.0208 | 0.0170 | 0.0107 |
| memorial | 0.0605 | 0.0429 | 0.1116 | 0.0973 | 0.0345 | 0.0054 | 0.0308 | 0.0016 | 0.0321 | 0.0028 | 0.0362 | 0.0032 |
| MtTamWest | 0.0053 | 0.0024 | 0.0449 | 0.0121 | 0.0468 | 0.0009 | 0.0175 | 0.0081 | 0.0406 | 0.0300 | 0.0205 | 0.0065 |
| rend06 | 0.0159 | 0.0037 | 0.0463 | 0.0322 | 0.0204 | 0.0016 | 0.0396 | 0.0016 | 0.0411 | 0.0025 | 0.0518 | 0.0030 |
| rosette | 0.0056 | 0.0035 | 0.0558 | 0.0062 | 0.0640 | 0.0008 | 0.0106 | 0.0023 | 0.0127 | 0.0044 | 0.0127 | 0.0025 |
| StillLife | 0.0221 | 0.0029 | 0.0507 | 0.0190 | 0.0355 | 0.0007 | 0.0247 | 0.0009 | 0.0258 | 0.0020 | 0.0316 | 0.0022 |
| Tree | 0.0434 | 0.0152 | 0.0844 | 0.0622 | 0.0435 | 0.0029 | 0.0213 | 0.0045 | 0.0216 | 0.0066 | 0.0223 | 0.0022 |
| Reinhard(global) | Reinhard(local) | Drago | Fattal | Shan | Gu | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Images | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. |
| Apartment_float | 0.8193 | 0.2674 | 0.4721 | 0.3147 | 0.4177 | 0.1720 | 0.2542 | 0.1514 | 0.4268 | 0.2959 | 0.2780 | 0.1281 |
| dani_belgium | 0.5589 | 0.3940 | 0.9748 | 0.7122 | 0.9940 | 0.4174 | 0.6488 | 0.3175 | 0.9644 | 0.6987 | 0.6961 | 0.3003 |
| Desk | 2.6314 | 0.6395 | 3.3420 | 1.3861 | 3.9240 | 0.9566 | 2.0510 | 0.7582 | 2.1173 | 1.2641 | 2.9466 | 0.7737 |
| Display1000_float | 0.4630 | 0.2664 | 0.9950 | 0.7335 | 0.6737 | 0.2687 | 0.4691 | 0.2634 | 0.6964 | 0.4213 | 0.5600 | 0.2615 |
| memorial | 3.2015 | 0.4858 | 1.8574 | 0.7741 | 3.5305 | 0.2863 | 3.8368 | 1.5534 | 4.5643 | 1.3001 | 4.8098 | 1.6655 |
| MtTamWest | 1.0522 | 0.5513 | 4.0217 | 1.8243 | 6.9016 | 1.9800 | 1.1497 | 0.2810 | 1.4351 | 0.5257 | 1.8670 | 0.2307 |
| rend06 | 1.9946 | 0.5657 | 2.0995 | 1.4428 | 1.7591 | 1.1248 | 8.4673 | 3.7136 | 9.3332 | 2.8647 | 13.5360 | 3.5505 |
| rosette | 0.7701 | 0.4208 | 10.3067 | 4.9597 | 18.1123 | 2.9129 | 0.8894 | 0.9286 | 0.8983 | 0.9128 | 1.1258 | 0.9651 |
| StillLife | 7.9052 | 2.5178 | 6.8135 | 2.8522 | 11.3119 | 4.9404 | 6.8368 | 3.8103 | 6.6331 | 3.5000 | 9.9990 | 4.8098 |
| Tree | 6.4565 | 0.5583 | 3.2179 | 1.0439 | 5.6125 | 0.9509 | 2.1995 | 0.7863 | 1.9026 | 0.7583 | 2.7228 | 0.8890 |
| Reinhard(global) | Reinhard(local) | Drago | Fattal | Shan | Gu | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Images | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. | Conv. | Prop. |
| Apartment_float | 0.8024 | 0.8029 | 0.8235 | 0.8241 | 0.7935 | 0.7941 | 0.7887 | 0.7887 | 0.7309 | 0.7310 | 0.8892 | 0.8894 |
| dani_belgium | 0.9219 | 0.9214 | 0.9011 | 0.9009 | 0.9072 | 0.9072 | 0.8542 | 0.8542 | 0.7368 | 0.7369 | 0.9489 | 0.9508 |
| Desk | 0.9609 | 0.9535 | 0.9495 | 0.9407 | 0.9604 | 0.9443 | 0.8921 | 0.8815 | 0.7521 | 0.7510 | 0.8272 | 0.8335 |
| Display1000_float | 0.9551 | 0.9537 | 0.8609 | 0.8596 | 0.9370 | 0.9363 | 0.7882 | 0.7873 | 0.7316 | 0.7311 | 0.8795 | 0.8782 |
| memorial | 0.8271 | 0.8475 | 0.8563 | 0.8601 | 0.8325 | 0.8590 | 0.7978 | 0.7909 | 0.7894 | 0.7828 | 0.8553 | 0.8701 |
| MtTamWest | 0.9673 | 0.9643 | 0.8936 | 0.8797 | 0.9518 | 0.9185 | 0.8052 | 0.8039 | 0.7798 | 0.7795 | 0.9267 | 0.9347 |
| rend06 | 0.9491 | 0.9441 | 0.8149 | 0.8108 | 0.9385 | 0.9277 | 0.7873 | 0.7758 | 0.7840 | 0.7644 | 0.9218 | 0.8241 |
| rosette | 0.8265 | 0.8231 | 0.8469 | 0.8304 | 0.8136 | 0.7566 | 0.7968 | 0.7936 | 0.7632 | 0.7605 | 0.8741 | 0.8704 |
| StillLife | 0.8825 | 0.8436 | 0.8211 | 0.7996 | 0.8611 | 0.8117 | 0.7272 | 0.7284 | 0.7423 | 0.7384 | 0.8230 | 0.8011 |
| Tree | 0.9609 | 0.9430 | 0.9212 | 0.9117 | 0.9641 | 0.9473 | 0.8499 | 0.8407 | 0.7748 | 0.7701 | 0.8340 | 0.8381 |
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
\field
A
\authorlist\authorentryYuma KINOSHITAsA\MembershipNumber1614118 \authorentryKouki SEOnA\MembershipNumber \authorentryArtit VISAVAKITCHAROENnA\MembershipNumber \authorentry[[email protected]]Hitoshi KIYAfA\MembershipNumber7904021
\affiliate[A]The authors are with Department of Information and Commu nication Systems, Tokyo Metropolitan University, Hino-shi, 191-0065 Japan.
11 11
A hue-preserving tone mapping scheme
based on constant-hue plane without gamut problem
(2019; 2019)
keywords:
Tone Mapping, Color Correction, Maximally Saturated Color, Hue Preservation
{summary}
We propose a novel hue-preserving tone mapping scheme. Various tone mapping operations have been studied so far, but there are very few works on color distortion caused in image tone mapping. First, LDR images produced from HDR ones by using conventional tone mapping operators (TMOs) are pointed out to have some distortion in hue values due to clipping and rounding quantization processing. Next,we propose a novel method which allows LDR images to have the same maximally saturated color values as those of HDR ones. Generated LDR images by the proposed method have smaller hue degradation than LDR ones generated by conventional TMOs. Moreover, the proposed method is applicable to any TMOs. In an experiment, the proposed method is demonstrated not only to produce images with small hue degradation but also to maintain well-mapped luminance, in terms of three objective metrics: TMQI, hue value in CIEDE2000, and the maximally saturated color on the constant-hue plane in the RGB color space.
1 Introduction
The interest of high dynamic range (HDR) imaging has recently been increasing in various area: photography, medical imaging, computer graphics, on vehicle cameras, astronautics. HDR images have the information of the wide dynamic range of real scenes. However, commonly used display devices having low dynamic range (LDR) cannot express the information that HDR images have. In contrast, display devices which can directly represent HDR images are not popular yet. Therefore, the importance of tone mapping (TM) operations has been growing in order to display HDR images on conventional LDR devices.
Tone mapping (TM) operation generates LDR images from HDR ones to display HDR ones on conventional LDR devices. Various research works on TM have so far been reported [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]. Most conventional studies have focused on compressing the luminance range of HDR images. In these methods, the range of the luminance of an HDR image is compressed by using a TM operator (TMO), and then an LDR image is produced by combining the compressed luminance and the color information of the original HDR image. However, TM operations only focusing on the luminance cause image colors to be distorted as pointed out in [14]. Mantiuk et al. [14] showed that the color distortion in tone-mapped LDR images occur due a tone curve, and they proposed a color correction formula on the basis of experimental results illustrating a relationship between the contrast-compression ratio and the saturation of the image. However, they only focused on tone curves as the cause of the color distortion, and other causes have never been considered.
In this paper, we first point out that color distortion in tone-mapped images is primarily caused by clipping and quantizing pixel values of tone-mapped LDR images. Then, we propose a novel hue-preserving tone mapping scheme based on constant-hue plane in the RGB color space. The proposed method consists of two steps: tone mapping with an conventional method and compensating hue distorted by the tone mapping. The proposed compensation method utilizes the hue information based on the maximally saturated colors [15, 16], for suppressing color distortions due to the tone mapping. The compensation is done by replacing the hue information of a tone-mapped LDR image with that of the original HDR image. Hue-preserving image-processing methods have already been developed in various research areas such as image enhancement and noise reduction [15, 17, 18, 19, 20, 21, 22]. These methods perform some operations under the conditions that the hue is fixed. In contrast, there are very few hue-preserving TM operations [14], and the proposed hue compensation method is independently performed from TM. This approach enables us not only to reduce color distortion, but also to maintain advantage of conventional TMOs.
To evaluate the effectiveness of the proposed method, we performed a number of simulations. In the simulations, the proposed method was compared with conventional TM operations in terms of the hue difference used in CIEDE 2000 [23] and the maximally saturated color difference. Experimental results showed that the hue difference between tone-mapped LDR images and the original HDR images can be reduced by applying the proposed method. Moreover, results of objective quality evaluation with the tone mapped image quality index (TMQI) [24] illustrated that the proposed method can maintain the performance of conventional TMOs.
2 PREPARATION
A general TM operation and the constant-hue plane in the RGB color space are summarized, here.
2.1 Tone mapping
A general TM operation is summarized, here. A TM operation consists of the following four steps.
(a)
The world luminance of an HDR image is calculated from RGB pixel values of the HDR image as,
[TABLE]
where and are RGB pixel values of the HDR image with a pixel , respectively.
(b)
The display luminance is calculated by using a TMO.
(c)
The floating-point pixel values of the LDR image is calculated as follows:
[TABLE]
where is the floating-point RGB value of the input HDR image , and . Besides, the gamma correction is performed for as needed.
(d)
The 8-bit color RGB values of the LDR image is derived from
[TABLE]
where rounds to its nearest integer value, and . Further, the value of is redefined as follows:
[TABLE]
where if takes a value over 255, it is clipped at 255. Also, if takes a negative value, it is clipped at 0. The rounding quantization in Eq.(3) and clipping in Eq.(4) generate some error.
In the case of ”Photographic Tone Reproduction” [1] which is a typical TMO, the display luminance in step(b) is calculated in accordance with the following procedure. The geometric mean of the world luminance is calculated as follows:
[TABLE]
where is the total number of pixels in the input HDR image . The scaled luminance is calculated as
[TABLE]
where is the parameter called “key value”, which indicates subjectively if the scene is light, normal, or dark. The display luminance is calculated by using the TMO as follows:
[TABLE]
2.2 Constant hue plane in the RGB color space
We focus on the constant hue plane in the RGB color space [15] to discuss color distortion. An input image is a 24-bit full color image and each pixel of the image is represented as . and are the R, G, and B components of the pixel , respectively, as shown in Fig.1. In the RGB color space, a set of pixels which has the same hue forms a plane, called constant hue plane. The shape of the constant hue plane is the triangle whose vertices correspond to white, black and the maximally saturated color, where , and are white, black and the maximally saturated color with the same hue as , respectively. The maximally saturated color is calculated by
[TABLE]
[TABLE]
[TABLE]
where and are functions that return the maximum and minimum elements of the pixel , respectively. Therefore, from Eq.(8), the elements of corresponding to the maximum and minimum elements of the pixel become 1 and 0, respectively.
On the constant hue plane, a pixel can be represented as a linear combination as
[TABLE]
where
[TABLE]
[TABLE]
[TABLE]
Since and exist on the plane and is an interior point of and , the following equations hold.
[TABLE]
[TABLE]
3 PROPOSED HUE COMPENSATION
We propose a method for compensating hue values of tone-mapped images.
3.1 Hue distortion
Hue distortion occurs due to the influence of the following three operations.
a)
Tone curve in step (b)
b)
Rounding quantization in Eq.(3)
c)
Clipping in Eq.(4)
Mantiuk et al. pointed out that a) tone curve generates color distortion on the CIECAM02 color appearance model, and proposed a correction formula [14]. However, they have never pointed out the influence of the operations b) and c).
The framework of the proposed method is shown in Fig.2. As illustrated in Fig.2, the proposed method can be applied to any TMO and moreover the influence of the above three operations can be considered.
Figure 3 shows an example of the difference of the maximally saturated colors between an original HDR image and the tone-mapped LDR image including the influence of clipping and rounding quantization processing. Figure 4 shows an example of the maximally saturated color of an original HDR image and the tone-mapped LDR image. In these figures, the influence of rounding quantization and clipping is demonstrated to cause color distortion.
3.2 HDR images on the constant hue plane
We consider giving HDR images the same pixel representation on the constant hue plane as LDR ones described in 3.1.
Each pixel of an HDR image is represented as where and are generally real numbers. As well as LDR images, we define white, black and maximally saturated color as , and . Maximally saturated color of the HDR image is calculated by replacing with in Eq.(8), and then the pixel value is represented as a linear combination as
[TABLE]
where and are coefficients which are calculated in accordance with Eq.(10). For HDR images, and exist on the same plane in the RGB color space, but is not always an interior point of and . That is, and do not meet Eq.(12), although they satisfy Eq.(11).
The proposed method aims to suppress some hue distortion included in LDR images mapped from HDR ones. Now, let be a pixel value of a tone-mapped LDR image that includes the influence of rounding quantization and clipping processing. In accordance with Eq.(9), the pixel value is expressed as follows
[TABLE]
where is a maximally saturated color calculated from the pixel value by Eq.(8). and are coefficients calculated by Eq.(10). Here, in general, is not met due to the above reasons. Therefore, to correct the hue of , we replace with as,
[TABLE]
Note that and are on the same constant hue plane.
3.3 Proposed procedure
The procedure of the proposed tone mapping scheme is summarized as follows.
(1)
Generate an LDR image from an HDR image by using a conventional TMO.
(2)
Calculate the coefficients of the generated LDR image for each pixel value in accordance with Eq.(10).
(3)
Calculate the maximally saturated color of the original HDR image by using Eq.(8).
(4)
Calculate the pixel value of the compensated LDR image in accordance with Eq.(15).
The R, G and B components of are in the range [0, 1] from Eq.(8). Therefore, the R, G and B components of are also in the range [0, 1] even after compensation. This allows us to prevent clipping error.
The proposed scheme is applicable to any conventional TMO. In addition, it can consider all influences caused in three operations described in 3.1.
4 Simulation
In an experiment, the proposed scheme was compared with conventional TMOs without any compensation and Mantiuk’s hue correction formula.
4.1 Simulation condition
We used 10 HDR images selected from an HDR image database [25] for the evaluation. The following is the procedure of the evaluation.
(1)
Generate LDR images from prepared HDR images by using four TMOs: Reinhard’s Global Operator [1], Reinhard’s Local Operator [1], Drago’s TMO [2], Fattal’s TMO [3], Shan’s TMO[4] and Gu’s TMO[5].
(2)
Apply the proposed method to , and obtain compensated LDR images .
(3)
Compare with and images corrected by Mantiuk’s method.
4.2 Simulation results
A Hue distortion
We used two objective metrics to evaluate hue distortion caused in TM operations. One is the difference in the maximally saturated colors between image with pixels and with pixels. it is calculated by the following three steps.
(1)
Calculate the maximally saturated colors from a pixel value in and in , from Eq.(8), where indicates the -th pixel of images , , and is its the maximally saturated color.
(2)
The difference between and for each pixel is given by
[TABLE]
where is euclidean norm.
(3)
Calculate the average value of over all pixels.
The other is the hue differences in CIEDE2000, which was published by the CIE [23]. The difference of hue values between two images was calculated as for each pixel, and as the average value of all pixels .
Tables 1 and 2 show the evaluation results in terms of the objective metrics, where is an HDR image and is a tone-mapped image. ”Conventional” indicates that images were generated by a conventional TMO without any hue compensation. In the tables, the proposed method outperformed the conventional approach for all TMOs. Therefore, the proposed method is effective for improving hue distortion included in fused images in terms of not only but also . Note that for the proposed method does not become zero value. This is because the rounding quantization has to be carried out again to generate integer pixel values, although clipping does not be done. Figures 5, 6 and 7 show some examples of the simulation results.
B Influence on tone mapping
Next, we evaluated the quality of tone-mapped LDR images in terms of Tone Mapped image Quality Index (TMQI) [24], which is a well-known objective quality assessment algorithm for tone-mapped images by combining a multi-scale signal fidelity measure based on a modified structural similarity (SSIM) [26] index and a naturalness measure based on intensity statistics of natural images. A higher TMQI score indicates higher quality.
From Table 3, the proposed scheme is shown to offer almost the same TMQI scores as those of the conventional approach, but many of the scores slightly decreased, due to the change of luminance caused by using the proposed method. In contrast, the change of luminance is easily corrected by adjusting the mean luminance of an image by multiplying all luminance values by a constant. Figure 8 shows the image of ”StillLife” in which TMQI is remarkably low compared with the conventional method. By comparing Fig.8(b) with Fig.8(c), it is confirmed that the slight change of luminance caused by applying the proposed method was easily corrected by the simple post-processing, so the TMQI score was improved, while maintaining and .
4.3 Comparison with Mantiuk’s method
Finally, our method was compared with Mantiuk’s hue correction formula [14]. We generated three LDR images by using Durand’s TMO [6] as in [14], and both our method and Mantiuk’s method were applied to the images respectively. Table 4 shows the comparison result. From the table, the our method is confirmed to outperform Mantiuk’s method in terms of both and . This is because Mantiuk’s method does not consider the influence of clipping and rounding quantization processing as described in 3.1. In contrast, the our method can compensate hue distortion caused by three operations.
5 Conclusion
In this paper, we proposed a novel TM scheme that utilize maximally saturated colors calculated from original HDR image on the basis of the constant hue plane in the RGB color space. The proposed method enables us to compensate the hue distortion of the tone-mapped LDR image caused by a conventional TMO, while maintaining TM performances that conventional TMOs have. Simulation results showed the effectiveness of the proposed method in terms of three objective metrics: , and TMQI. Moreover, our method was compared with Mantiuk’s method to show the effectiveness.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda, “Photographic tone reproduction for digital images,” ACM Trans. Graph. (TOG), vol.21, no.3, pp.267–276, 2002.
- 2[2] F. Drago, K. Myszkowski, T. Annen, and N. Chiba, “Adaptive Logarithmic Mapping For Displaying High Contrast Scenes,” Computer Graphics Forum, vol.22, no.3, pp.419–426, 2003.
- 3[3] R. Fattal, D. Lischinski, and M. Werman, “Gradient domain high dynamic range compression,” ACM Trans. Graph., vol.21, no.3, pp.249–256, July 2002.
- 4[4] Q. Shan, J. Jia, and M.S. Brown, “Globally optimized linear windowed tone mapping,” IEEE Transactions on Visualization and Computer Graphics, vol.16, no.4, pp.663–675, July 2010.
- 5[5] B. Gu, W. Li, M. Zhu, and M. Wang, “Local edge-preserving multiscale decomposition for high dynamic range image tone mapping,” IEEE Transactions on Image Processing, vol.22, no.1, pp.70–79, Jan 2013.
- 6[6] F. Durand and J. Dorsey, “Fast bilateral filtering for the display of high-dynamic-range images,” ACM Trans. Graph., vol.21, no.3, pp.257–266, July 2002.
- 7[7] E. Sikudová, T. Pouli, A. Artusi, A.O. Akyüz, F. Banterle, Z.M. Mazlumoglu, and E. Reinhard, “A gamut-mapping framework for color-accurate reproduction of hdr images,” IEEE Computer Graphics and Applications, vol.36, no.4, pp.78–90, July 2016.
- 8[8] A. Artusi, T. Richter, T. Ebrahimi, and R.K. Mantiuk, “High dynamic range imaging technology [lecture notes],” IEEE Signal Processing Magazine, vol.34, no.5, pp.165–172, Sep. 2017.
