Objective Image Quality Assessment for High Resolution Photospheric Images by Median Filter Gradient Similarity

TL;DR

This paper introduces a new no-reference image quality metric called Median Filter Gradient Similarity (MFGS) for photospheric images, demonstrating its effectiveness in assessing solar telescope images with high correlation to traditional contrast measures.

Contribution

The paper proposes MFGS, a novel no-reference image quality metric for photospheric images, validated on solar telescope data, showing its robustness and correlation with RMS-contrast.

Findings

MFGS value decreases monotonically with image degradation.

MFGS correlates linearly with RMS-contrast.

MFGS is less affected by image content variations.

Abstract

All next generation ground-based and space-based solar telescopes require a good quality assessment metric in order to evaluate their imaging performance. In this paper, a new image quality metric, the median filter gradient similarity (MFGS) is proposed for photospheric images. MFGS is a no-reference/blind objective image quality metric (IQM) by a measurement result between 0 and 1 and has been performed on short-exposure photospheric images captured by the New Vacuum Solar Telescope (NVST) of the Fuxian Solar Observatory and by the Solar Optical Telescope (SOT) onboard the Hinode satellite, respectively. The results show that: (1)the measured value of MFGS changes monotonically from 1 to 0 with degradation of image quality; (2)there exists a linear correlation between the measured values of MFGS and root-mean-square-contrast (RMS-contrast) of granulation; (3)MFGS is less affected by the image contents than the granular RMS-contrast. Overall, MFGS is a good alternative for the quality assessment of photospheric images.

Figures

Tables

Table 1. Table 1. : Four sets of sample images taken with NVST. The seeing parameter r 0 subscript 𝑟 0 r_{0} was calculated by the method developed by  von der Luhe ( 1984 ) . \ilabel T-dataset

Dataset	Start time (UT)	Active region number	Location	$r_0$ (cm)
A	04:51:11, 26 Sep. 2012	NOAA 11575	W17N08	12.1
B	02:15:02, 29 Oct. 2012	NOAA 11598	S11W27	9.4
C	04:33:59, 26 Oct. 2012	NOAA 11598	S12E17	8.5
D	02:51:31, 29 Oct. 2012	NOAA 11598	S11W37	8.2

Table 2. Table 2. : The average values of MFGS derived for different datasets by applying different gradient operators. \ilabel T-Operators

	SOT	A	B	C	D
Roberts	0.991	0.837	0.783	0.751	0.726
Prewitt	0.998	0.984	0.966	0.952	0.938
Sobel	0.998	0.981	0.960	0.945	0.930
difference	0.984	0.710	0.616	0.590	0.552

Table 3. Table 3. : Correlation coefficients between MFGS and RMS-contrast metrics. \ilabel T-Correlation

	A	B	C	D
Full image	0.862	0.571	0.692	0.867
Active region area	0.769	0.571	0.739	0.878
Quiet-Sun area	0.932	0.889	0.902	0.968

Table 4. Table 4. : Correlation coefficients between the metric values obtained from active region areas and quiet-Sun areas by MFGS and RMS-contrast. \ilabel T-dependence

	A	B	C	D
MFGS	0.823	0.798	0.700	0.978
RMS-contrast	0.758	0.567	0.547	0.896

Equations

$$P=M\left(R,[33]\right),\text{ilabel}eq-filtering$$

$$G_{\mathrm{r}}=\sum |D(R)|,\text{ilabel}eq-gradient1$$

$$G_p=\sum |D(P)|,\text{ilabel}eq-gradient2$$

$$\text{MFGS}=\left(2G_{\mathrm{p}}{G}_{\mathrm{r}}\right)/\left(G_{\mathrm{p}}^2+G_{\mathrm{r}}^2\right).\text{ilabel}eq-mfgs$$

$$\text{rms-contrast}=\sqrt{\frac{1}{N}\sum _{i=1}^N{\left(I-\bar{I}\right)}^2}/\bar{I},$$

Full text

\setlastpage\inarticletrue

{opening}

1 Introduction

\ilabel

S-Introduction The quality of the images captured by ground-based telescopes is heavily limited by the Earth’s atmospheric turbulence which is commonly termed as “seeing”. For telescopes without Adaptive Optics (AO) systems, to alleviate the image degradation induced by the atmospheric turbulence and to achieve higher angular resolution, post-facto reconstruction techniques (e.g. speckle image reconstruction) are widely used. Even for those telescopes with AO systems, the observed images are often processed post-facto to further reduce residual aberrations and to achieve the diffraction limit of the telescope over a larger field of view (FOV). Under such circumstances, a proper objective image quality metric (IQM) is required to evaluate the quality of the short-exposure images (frames) captured by high-speed cameras. The frames with higher values of such a metric might be selected for the subsequent reconstruction process. Namely, an improved IQM will be utilized to streamline the post-facto image processing.

Based on the amount of information required, objective IQMs can be classified into full-reference (FR), no-reference (NR)/blind and reduced-reference (RR) ones. FR IQMs require the entire reference image to be available, usually without blemishes. The mean squared error (MSE) is the simplest FR IQM, computed by averaging the squared intensity differences of distorted and reference image pixels. The structural similarity (SSIM) proposed by Wang et al. (2004) is one of the most widely used FR IQMs because its measurement result is close to the result given by human visual system. In many practical applications (e.g. astronomical observations), the reference image is not well-defined, so that NR/blind IQMs are preferred. Some NR IQMs have been studied or used in astronomical observation and image processing applications. The sharpness of an image measured after high pass filtering was used for real-time frame selection by the Swedish Vacuum Solar Telescope (Scharmer, 1989). Bos and Roggemann (2012) compared the effect of sharpness and entropy in tuning the inverse filter used for amplitude recovery in a speckle imaging system. The Fisher information is a measure of disorder and was used by Zhang, Suess, and Mackay (2006) for searching lucky images. The Fourier amplitude was used by Garrel, Guyon, and Baudoz (2012) for frame selection.

For the quality assessment of high resolution photospheric images, the most commonly used is the root-mean-square-contrast (RMS-contrast) (Denker et al., 2005, 2007; Danilovic et al., 2008; Scharmer et al., 2010). When the measure of RMS-contrast is only applied to the quiet-Sun region of a photospheric image, it is called the granular RMS-contrast. One shortcoming of the granular RMS-contrast is that it depends on the wavelength used (Albregtsen and Hansen, 1977; Ricort and Aime, 1979) and also on the distance from the disk center (Cuberes et al., 2000); with the increasing heliocentric angle or observing wavelength, the granular RMS-contrast decreases (see Figures 1 and 2 in Albregtsen and Hansen (1977)). Moreover, the RMS-contrast value of an image depends strongly on the structural contents of the assessed image. When dark features (pores or sunspots) are moving in and out of the field of view (FOV), the value of RMS-contrast must be interpreted carefully.

In this paper, we present the median filter gradient similarity (MFGS), a new NR objective IQM. As you see below, MFGS shows advantages in measuring the quality of photospheric images over e.g. RMS-contrast. The rest of this paper is organized as follows. In Section \irefS-mfgs, we introduce MFGS. In Section \irefS-Experiment, the performance of MFGS on photospheric images is validated. In Section \irefS-Comparison, MFGS and RMS-contrast are compared by looking at their mutual correlation and their dependences on the image contents. In Section \irefS-ConclusionDiscussions, discussion and conclusion are given.

2 Median Filter Gradient Similarity

\ilabel

S-mfgs

2.1 Principle of MFGS

\ilabel

sub-mfgsPrinciple The MFGS metric originated from both the SSIM assessment method (Wang et al., 2004) and the NR perceptual blur metric (Crete et al., 2007). SSIM was designed to improve the traditional algorithms like peak signal-to-noise ratio and mean squared error, which have been proven to be inconsistent with human eye perception. The main objective of the SSIM metric is to evaluate the similarity between the assessed image and its reference one in terms of luminance, contrast, and structure. However, SSIM is a FR metric and cannot be directly used in astronomical applications. The NR perceptual blur metric is based on the property that it is difficult for a human being to perceive differences between a blurred image and its re-blurred one.

Since the observation of small-scale structures in the photosphere is one of the main scientific goals for solar telescopes, structural information is a key factor that we must consider when proposing a new IQM. In this aspect, the pre-processing of a raw image by a noise removal filter enhances the effectiveness of IQM when the image quality is low.

Based on these considerations, we propose MFGS, a new image quality assessment metric for photospheric images. The basic process of MFGS consists of (1) filtering the raw image (image under evaluation) to obtain a processed image (reference image); (2) calculating the gradients of the raw and the processed images, respectively; (3) calculating the similarity between the two gradients. The flowchart is shown in Figure \irefF-MFGS_2a.

2.2 Algorithm of MFGS

\ilabel

sub-mfgschart As was stated above, MFGS contains three steps, i.e., filtering the raw image, calculation of the gradient, and calculation of the similarity. They are described below in detail.

2.2.1 Step 1 - Filtering the Raw Image with a Median Filter

A median filter, a nonlinear operation, is more effective than convolution in order to reduce noise and preserve edges simultaneously (Brownrigg, 1984; Lin and Willson Jr, 1988; Sun and Neuvo, 1994; Wang and Zhang, 1999; Arias-Castro and Donoho, 2009). A 3 $\times$ 3 median filter is applied to the raw image ($R$) to obtain the processed image ($P$):

[TABLE]

where $M$ is the two-dimensional (2-D) median filtering operation, and [3 3] is a 3 $\times$ 3 local window.

2.2.2 Step 2 - Calculating the Gradient Value of $R$ and $P$

Structural information in the images is represented by the gradient, which is an intuitive and simple measure of structural distortion and is widely used in objective image quality assessment (Asatryan and Egiazarian, 2009; Liang et al., 2010; Kim, Han, and Park, 2010; Zhu and Wang, 2012). Two gradient values, $G_{\rm r}$ for the raw image and $G_{\rm p}$ for the processed image, are obtained as

[TABLE]

[TABLE]

where $D$ is the gradient operator of difference (\left[\begin{array}[]{cc}-1&1\end{array}\right]).

2.2.3 Step 3 - Calculating the Similarity between $G_{\rm r}$ and $G_{\rm p}$

The following equation, which has been widely used in similarity-based image quality assessments (Wang, Bovik, and Lu, 2002; Wang et al., 2004; Chen, Yang, and Xie, 2006; Rehman and Wang, 2012; Zhu and Wang, 2012), is adopted to calculate the similarity between R and P,

[TABLE]

Obviously, MFGS takes a value from 0 to 1. The quality of the image is better if its MFGS value is higher; a perfect image is represented by MFGS = 1.

3 Performance Evaluation

\ilabel

S-Experiment

3.1 Sample Data

\ilabel

S-Experimentdata Sample images were taken with the New Vacuum Solar Telescope (NVST; Liu and Xu (2011); Liu et al. (2014)). NVST is a vacuum solar telescope with 1 m aperture that aims to observe fine structures on the Sun. It is the main observation facility of the Fuxian Solar Observatory (FSO) located at $24^{\circ}34^{\prime}48^{\prime\prime}$N, $102^{\circ}57^{\prime}01^{\prime\prime}$E, on the northeast side of the Fuxian Lake, Yunnan, China. Its high resolution imaging system consists of one chromosphere channel (H$\alpha$, 6563 Å) and two photosphere channels (TiO band, 7058Å, and G-band, 4300Å).

The sample photospheric images were obtained at the wavelength of 7058 Å by a high-speed CMOS camera (10-15 frames per second, about 1 millisecond exposure time). The FOV was $102^{\prime\prime}\times 86^{\prime\prime}$ with an image scale of 0.04*′′*/pixel (2560 $\times$ 2160 pixels). All the images were taken without AO, and basic image corrections (dark subtraction and flat fielding) were made on them.

Four representative data sets taken in September and October 2012 were selected from a huge amount of observational data sets. The four data sets include two active regions, NOAA 11575 and 11598, which represented observations with different quality levels. Each set consists of 200 short-exposure frames taken with a cadence of about 15 s. The seeing parameter $r_{0}$ of each set was calculated by the method developed by von der Luhe (1984). The four sets are labeled as “A”, “B”, “C”, and “D” in the descending order of $r_{0}$ values (Table \irefT-dataset).

In addition, a G-band image taken with the Solar Optical Telescope (SOT; Tsuneta et al. (2007)) onboard the Hinode satellite on 28 February 2007 was used to further confirm the performance of MFGS on images without atmospheric turbulence. The image of perfect quality (see “e” in Figure \irefmfgs-High) was taken with an exposure time of 51 milliseconds. The FOV was $56^{\prime\prime}\times 56^{\prime\prime}$ with an image scale of $0.05^{\prime\prime}$/pixel.

3.2 Choice of Gradient Operators

\ilabel

S-gradient operators Gradient calculation is the core in the second step of the MFGS algorithm. There are several operators to obtain the gradient approximation, e.g., the Sobel \left[\begin{array}[]{ccc}-1&0&1\\ -2&0&2\\ -1&0&1\end{array}\right], Prewitt \left[\begin{array}[]{ccc}-1&0&1\\ -1&0&1\\ -1&0&1\end{array}\right], Roberts \left[\begin{array}[]{cc}0&1\\ -1&0\end{array}\right] and the difference \left[\begin{array}[]{cc}-1&1\end{array}\right] operators. We conducted an experiment to find out which operator is most suitable for MFGS. The four sets listed in Table \irefT-dataset and the image taken with Hinode/SOT were processed by MFGS, by applying the Sobel, Prewitt, Roberts, and difference operators. The calculated average values of MFGS are listed in Table \irefT-Operators. One can see that the discrimination power of the Roberts and difference operators is superior to the Prewitt and Sobel operators. The SOT image was taken without atmospheric turbulence and its quality should be much better than any short-exposure images taken by the ground-based NVST. However, when performing MFGS with the Prewitt or Sobel operator, the obtained values for the SOT image are very close to those from NVST. Moreover, the difference operator is faster in computation time than the Roberts operator. Therefore, the difference operator is the most suitable one among the examined operators for MFGS.

3.3 Performance of MFGS on Sample Data

\ilabel

Sub-Experimentresults MFGS was applied to each image frame in the data sets listed in Table \irefT-dataset. Figure \irefmfgs-full shows the variations in the MFGS values versus image index (i.e., time). Overall, all of the MFGS values are between 0.5 and 0.8; the average values for sets A, B, C, and D are 0.710, 0.616, 0.590, and 0.552, respectively. These values are consistent in the order with their r0 values. In each data set, the MFGS value fluctuates among the frames, which reflects the fluctuation of image quality due to atmospheric turbulence.

The images with the highest value in each data set are shown in Figure \irefmfgs-High as labeled “a”, “b”, “c”, and “d”, respectively. One can see clearly that the change in their image quality agrees with their MFGS value well. The image taken with Hinode/SOT is also shown in Figure \irefmfgs-Highe. Its image quality is much better than the others and its MFGS value (0.984) is much higher.

Next, the images with the lowest value in each data set are shown in Figure \irefmfgs-Low as labeled “a”, “b”, “c”, and “d”, respectively. It is obvious that the change in their image quality also agrees with their MFGS value, and the overall quality of these images is much worse than the images in Figure \irefmfgs-High.

4 Comparison between RMS-Contrast and MFGS

\ilabel

S-Comparison In this section, MFGS and RMS-contrast are compared, particularly by paying attention to their dependence on the image properties. For this purpose, each image in the data sets given in Table \irefT-dataset was divided into two parts, i.e., the active region area and quiet-Sun region area; the latter included no sunspots nor pores. Then MFGS and RMS-contrast were applied separately to these two areas. The RMS-contrast value is defined as

[TABLE]

where $\bar{I}$ is the mean intensity and $N$ is the number of pixels (Roudier and Muller, 1986). The higher the quality of the image is, the bigger the value of the contrast would be.

4.1 Correlation between MFGS and RMS-Contrast

\ilabel

s-Correlation analysis Figure \irefMFGSvsRMS4 shows the correlation between MFGS and RMS-contrast metrics derived from the four image sets. The left panel shows that the images with good quality in data set A are erroneously determined as bad ones by the RMS-contrast metric. The middle panel shows that the images in data set B are erroneously determined as good ones by the RMS-contrast metric. However, the right panel displays a good linear correlation between MFGS and RMS-contrast metrics, which means that RMS-contrast and MFGS are tightly linearly correlated when performed on quiet-Sun structures (granules) with uniform and isotropic characteristics. Moreover, the variation in the RMS-contrast or MFGS values in the quiet-Sun areas agrees with the $r_{0}$ values better than those in the full images or active region areas.

Accordingly, we calculated the correlation coefficients between the MFGS and RMS-contrast metrics (Table \irefT-Correlation). The correlation coefficients for the quiet-Sun area (i.e., 0.932, 0.889, 0.902, 0.968) are higher than those for the full images or for active region areas. This also confirms the result shown in Figure \irefMFGSvsRMS4, i.e., a linear correlation between MFGS and RMS-contrast exists when performed on the quiet-Sun areas.

4.2 Dependence on Image Contents

\ilabel

s-Content dependence A good image quality metric should perform well, independent of image contents. In our case, the performance of a metric on the active region areas and the quiet-Sun areas can give some hint on this point. Therefore, we compared the metric values derived from the two areas. A weak correlation between the two values implies that the corresponding metric heavily depends on the image contents, and vice versa. Table \irefT-dependence shows the results for the MFGS and RMS-contrast metrics. For the RMS-contrast, the correlation coefficients are below 0.6 for data sets B and C, which indicates that the performance of the RMS-contrast metric depends significantly on the image contents. A photospheric image contains a variety of structures including granules, intergranular dark lanes, bright points, pores, and sunspots; rich in image contents. For the active region areas that contain pores and sunspots, the RMS-contrast value is dominated by the intensity differences among umbra, penumbra, and granules. With dark features (sunspots) moving in and out of the FOV, the RMS-contrast value will change significantly. For MFGS, the correlation coefficients are between 0.978 (maximum) and 0.700 (minimum), which indicates that the MFGS metric depends less on the image contents compared with the RMS-contrast metric. From this point of view, MFGS is better than RMS-contrast.

5 Discussion and Conclusions

\ilabel

S-ConclusionDiscussions We propose a new NR/blind objective image quality metric, i.e., the MFGS metric. The performance tests on short-exposure photospheric images acquired by NVST and the images acquired by Hinode/SOT show that the MFGS values change monotonically from 1 to 0 with the degradation of image quality. There exists a linear correlation between the derived values of MFGS and granular RMS-contrast. Moreover, MFGS is less affected by the image contents than RMS-contrast and it is a good alternative for the quality assessment of photospheric images.

There are issues that are worthy of discussion. First, all of the short-exposure sample images used in this study were obtained with NVST without its AO system. Although speckle image techniques have been successful in obtaining diffraction limited images, the most powerful technique for large astronomical telescopes is the one using AO technique. Therefore, an IQM is required to quantify the level of seeing compensation by the AO system. For MFGS is a good measure of photospheric image quality, we believe that it may be used in AO systems, but further investigation would be needed to validate it. Second, considering the fundamental principle of MFGS, we believe that it could also be used to assess images with rich structural information and sharp edges, like high resolution chromospheric images, but the expected performance should be checked through further studies.

Acknowledgements

This work is supported by the National Natural Science Foundation of China (11163004, U1231205, 11263004, 11303011, 11103005, 11463003 and 11203077) and Natural Science Foundation of Yunnan Province (2013FA013, 2013FA032, 2013FZ018 and 2013FZ018). Wenda Cao acknowledges the support of the US National Science Foundation (AGS-0847126, AGS-1146896). Hinode is a Japanese mission developed and launched by ISAS/JAXA, with NAOJ as domestic partner and NASA and STFC (UK) as international partners. It is operated by these agencies in co-operation with ESA and NSC (Norway). We thank the referee for providing valuable suggestions which substantially helped to improve the quality of the paper.