Show-through removal with sparsity-based blind deconvolution
Sota Kawakami, Hiroyuki Kudo, Abel C.H. Chen, Abel C.H. Chen, Abel C.H. Chen, Abel C.H. Chen
TL;DR
This paper introduces a new method to remove show-through in scanned documents using sparsity-based blind deconvolution techniques.
Contribution
A novel show-through removal method based on nuclear norm minimization and blind deconvolution is proposed.
Findings
The proposed method uses a cost function with data, nuclear norm, and regularization terms for effective show-through removal.
Simulation and real image experiments demonstrate the effectiveness of the new approach.
The method leverages the Accelerated Proximal Gradient and Singular Value Projection for fast convergence.
Abstract
When scanning a document printed on both sides by using an electronic scanner, the printed material on the back (front) side may be transmitted to the front (back) side. This phenomenon is called show-through. The problem to remove the show-through from scanned images is called the show-through removal in the literature. In this paper, we propose a new method of show-through removal based on the following principle. The proposed method uses two scanned images with the front side and with the back side as input images. The proposed method is based on Ahmed’s Blind Image Deconvolution method discovered in 2013, which succeeded in formulating Blind Image Deconvolution as a nuclear norm minimization. Since the structure of show-through removal resembles that of Blind Image Deconvolution, we discovered that the show-through removal can be reformulated into a nuclear norm minimization in the…
Genes, proteins, chemicals, diseases, species, mutations and cell lines named across the full text — each resolved to its canonical identifier and authoritative record.
Click any figure to enlarge with its caption.
Fig 1
Fig 2
Fig 3
Fig 4
Fig 5
Fig 6
Fig 7
Fig 8
Fig 9
Fig 10
Fig 11
Fig 12
Fig 13
Fig 14Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBlind Source Separation Techniques · Image and Signal Denoising Methods · Sparse and Compressive Sensing Techniques
1 Introduction
In recent years, digitization of paper documents has become very popular. When the paper documents are digitized, a phenomenon called show-through may occur, in which the printed material on the back (front) side is transmitted to the front (back) side. It reduces visibility of text and accuracy of OCR (Optical Character Recognition). The problem to remove the show-through from scanned images is called the show-through removal in the literature. Up to now, several methods of the show-through removal have been proposed. For example, the method using binarization [1, 2] and the method using color tone correction [3] have been used in electronic scanners. This is because the process of show-through removal in these methods is simple and requires only a single input image. However, these methods tend to be inaccurate for the printed matter having gradation such as images and drawings. On the other hand, other improved methods for the show-through removal have been proposed, in which both the scanned paper of front page and that of back page are used as input images. There exist a method using Kalman filter [4, 5], a method using Independent Component Analysis (ICA) [6–8], a method using Non-negative Matrix Factorization (NMF) [9], a method using Total Variation (TV) [10], and a method using adaptive filtering [11, 12]. These methods achieve higher accuracy compared to the earlier methods. However, each of these methods still suffers from drawbacks such as difficulty in adjusting parameters, inability to process well when there exists a printed matter on both the front and back pages, and dependency of the solution on initial values of iterative methods.
Also, in recent years, show-through removal methods using Convolutional Neural Network (CNN) have been proposed [13, 14]. Among them, the method based on Auto-encoder [14] has a large potential. A typical major application of Auto-encoder is to remove disturbance from signals or images, which has been already proposed in image denoising, speckle reduction in SAR or ultrasound images, and clutter rejection in radar images. The same method can be also used to remove show-through from scanned images. This method requires only a single image of front side. However, its accuracy depends on used training data, and it may be weaker to the show-through having strong correlation with the front page image. On the other hand, our proposed method requires both of an image of the fore side and that of the back side, but using the two images eliminates the necessity of training data and may lead to an improved accuracy (for example, in the above mentioned case). At the current stage, we believe that both the optimization-based approach like ours and the Auto-encoder-based approach are meaningful to study, because they have different advantages and disadvantages and the both may be able to be combined in the future to develop more accurate or more convenient methods.
In this paper, we propose a new method of show-through removal based on the nuclear norm minimization. The principle of this method is summarized as follows. First, the proposed method uses two scanned images with the front side and with the back side as input images. The proposed method is based on Ahmed’s Blind Image Deconvolution (BID) method [15] discovered in 2013. Most classical methods for BID were based on a non-convex optimization, but Ahmed et al. succeeded in formulating BID as a nuclear norm minimization, which is one of famous convex optimization problems appearing in Compressed Sensing (CS). The key of their work lies in reformulating BID in the space of outer product matrix constructed from an image vector and a Point Spread Function (P.S.F.) vector of blurring. Since the structure of show-through removal resembles that of BID, we discovered that the show-through removal can be reformulated into a nuclear norm minimization in the space of outer product matrix constructed from an image vector and a P.S.F. vector. We have succeeded in this reformulation by following the similar derivation step to Ahmed’s BID method. Using this key idea, we constructed the proposed method as follows. First, our cost function for minimization consists of the following three terms. The first term is the data term and the second term is the nuclear norm derived from the above reformulation. However, without a regularization term, the method does not work well due to the underdetermined nature of show-through removal problem and the existence of noise in the input images. Therefore, we introduce a regularization term, which is TV imposed on the images. The resulting nuclear minimization problem is solved by using Accelerated Proximal Gradient (APG) method [16] and Singular Value Projection (SVP) [17] with some problem-specific modifications, which converges fast and requires a simple implementation. In addition, the implementation of APG method is very simple.
The paper is organized as follows. In Section 2, we describe a mathematical model of show-through removal. In Section 3, we briefly review Ahmed’s BID method which is the basis of this work. In Section 4, we explain the proposed method in detail. The explanation mainly consists of 1) the derivation to convert the show-through removal into a nuclear norm minimization and 2) the algorithm description of the proposed method. In Section 5, we show results of simulation studies as well as results of real image experiments. The conclusion is described in Section 6.
2 Mathematical model of show-through removal
Before explaining the proposed method for show-through removal, we explain a mathematical model of image degradation by the show-through. Due to the show-through, pixel values of the original image on the front side are affected by the blurred image of the back side image and the transmittance (from the back side to the front side). The blurring is caused due to diffusion of light occurring when the light passes through the paper. Using equations, the standard model of show-through can be expressed as
where G1 is the image of front side with show-through, G2 is the image of back side with show-through, F1 is the original image of front side, F2 is the original image of back side. Furthermore, H is the P.S.F. of the above-mentioned blurring and the parameter a is the transmittance. Furthermore, M(∙) is the operator which flips the image horizontally and T(∙) is the linear operator which represents the degradation caused by the show-through. We note that this linear degradation model of show-through was described in [5].
Next, we describe the detailed form of operator T(∙) appearing in Eq (1). The operator T(∙) is expressed as
In Eq (2), B1(x,y) is the horizontally flipped blurred image of front side with inverted pixel value and gain a, B2(x,y) is the horizontally flipped blurred image of back side with inverted pixel value and gain a. Lmax is the maximum value of the image (255 for 8-bit images), and Kx and Ky are width and height of P.S.F. H, respectively. In addition, H(0,0) is defined as the center of H, not the first element. Also, we assume that F1(x,y) and M(F2)(x+x′,y+y′) (as well as F2(x,y) and M(F1)(x+x′,y+y′)) are uncorrelated to each other. In Fig 1, we show example images constructed by using this model. The show-through removal can be formulated as a problem to restore a pair of original images F1 and F2 from a pair of observed images G1 and G2 degraded by the show-through. We remark that, throughout this paper, both the scanned paper of the front side and that of the back side are available for the restoration, because using the two input images allows higher restoration accuracy as demonstrated in [4–12].
Example images constructed by using the show-through model.(a) G1 degraded image (front side), (b) G2 degraded image (back side), (c) F1 original image (front side), (d) F2 original image (back side), (e) B1 horizontally flipped blurred image with invert pixel and gain a (front side), (f) B2 horizontally flipped blurred image with invert pixel and gain a (back side).
3 Ahmed’s blind image deconvolution method
The proposed method for show-through removal is largely based on Ahmed’s BID method [15] discovered in 2013. In this section, we briefly review Ahmed’s BID method for unfamiliar readers. First, we explain image degradation model by the blurring. Let the original image be F and the P.S.F. of blurring be H. Then, the blurred image S can be expressed as
where the symbol ⨂ denotes two-dimensional (2-D) convolution. Let S(x,y),F(x,y), and H(x,y) be values of S,F, and H at pixel (x,y). Then, Eq (3) is expressed as
where Mx and My are width and height of F, respectively. BID is a problem of estimating the original image F and the P.S.F. H simultaneously from the degraded image S. Most conventional BID methods are based on a non-convex optimization by using an iterative method [18–20]. On the other hand, in 2013, Ahmed et al. [15] succeeded in formulating BID into a form of nuclear norm minimization, which is one of famous convex optimizations problems appearing in CS. The key of their work lies in reformulating BID in the space of outer product matrix constructed by the image vector and the P.S.F. vector. We briefly describe this derivation step below. Let be the vectorized form of F. Let be the vectorized form of H. We assume that the vectorization of image and P.S.F. is performed according to the raster-scan order. In Ahmed’s BID method, an outer product matrix X constructed from and is used, where X is defined by
By the definition of outer product, the rank of matrix X is always 1 when the two vectors and are non-zero. Then, by using X, we can perform the same calculation as Eq (4) representing the convolution by
where the convolution could be expressed with a simple linear operator which acts on the matrix X. We denote this linear operator by . Then, Eq (6) is expressed as
where is a vector defined by arranging all elements of image S according to the raster-scan order. Therefore, BID can be formulated as a problem to find a matrix X satisfying Eq (7) and having a rank of 1. This problem is expressed as
The problem of finding a matrix with rank of 1 is called the rank minimization in the literature. It is known that computational complexity of the rank minimization is NP-hard, but it can be solved with sufficient accuracy in practice by using a convex relaxation into a nuclear norm minimization. This fact has been demonstrated in a variety of problems in real world [21]. The nuclear norm minimization corresponding to Eq (8) is expressed as
Through the above derivation step, Ahmed et al. succeeded in formulating BID by a convex optimization. We remark again that the conventional methods of BID [18–20] are based on a non-convex optimization so that the solution depends on initial values of image and P.S.F. used in the iterative method. Therefore, we expect that Ahmed’s BID method allows more stable and more accurate restoration.
4 Proposed method for show-through removal
4.1 Application of Ahmed’s BID method to show-through removal
Let us consider the problem of show-through removal formulated in Section 2. The show-through removal method in this paper estimates the two original images F1 and F2 from the two input degraded images G1 and G2. First, in order to extend Ahmed’s BID method to the show-through removal, the definition of outer product X and the degradation model need to be modified as follows. We note that this modification is a key in this work. We define the outer product matrix X constructed from F1,F2,H,Lmax and a by
where is a vector defined by arranging all elements of images F1 and F2 (subtracted from Lmax) according to the raster-scan order. This vector takes the inversion of pixel values appearing in the show-through model of Eq (2) into account. Also, in the P.S.F. vector defined by arranging all elements of P.S.F. H according to the raster-scan order, the first KxKy elements of include the multiplication by the transmittance a. This definition allows us to embed both the degradation due to blur H and the effect of transmittance a simultaneously in . As a result, we no longer need to estimate the P.S.F. and the transmittance separately. Furthermore, KxKy+1-th (last) element of constructs KxKy+1-th column of matrix X which is equivalent to the image .
By defining the matrix X in this way, similarly to Eq (6), the degraded images G1 and G2 defined in Eq (2) can be calculated from the matrix X by
In Fig 2, we show the structure of matrix X and the process of generating G1 and G2 from the matrix X by Eq (11).
Furthermore, if we define the blurred image vector by linking the vectorized forms of G1 and G2, Eq (11) can be expressed by using the linear operator acting on the matrix X as
Schematic diagram of the structure of matrix XandthegenerationprocessofG1andG2frommatrixX.
According to the above derivation, we succeeded in modelling the image degradation by show-through using the outer product matrix X. Furthermore, from the definition of outer product, we know that the rank of matrix X is always 1 when the two vectors and are non-zero. So, by applying the same nuclear norm minimization as in Ahmed’s BID method, i.e. Eq (9) in the BID case, the show-through removal can be formulated as the following optimization problem.
4.2 Description of proposed algorithm
There exist mainly the following two drawbacks in original Ahmed’s BID method. The first drawback is lack of regularization term. Without the regularization term, it does not work well when the problem itself is underdetermined or when the degraded images contain noise. The second drawback is large computational cost, because Ahmed et al. used a slow and inefficient optimization method to solve Eq (9), i.e. general solver of semi-definite programming. To overcome these drawbacks, we have used the similar approach to that in our previous work on the standard BID and motionless super-resolution [22]. For the regularization, we have used a regularization term consisting of TV penalty imposed on the images. To reduce the computational cost, we have used APG method [16] and SVP [17] with some problem-specific modifications to solve Eq (13). This section describes the proposed algorithm in detail.
4.2.1 Regularization term
In its original form, Ahmed’s BID method used some unrealistic constraint such as a priori knowledge on the object support or a distribution of zero coefficient in some transformed (sparsified) space to overcome the problem of undetermined nature of BID problem, because the number of estimated variables is larger than the number of measured data in this problem. To overcome this drawback, we have proposed the use of regularization term [22]. Our regularization term consists of the TV penalty imposed on the images F1 and F2 to be estimated. Thanks to the regularization term described below, we expect that accuracy and stability of show-through removal can be improved significantly.
We denote the TV penalty imposed on the images F1 and F2 by R(X). It is well-known that TV possesses an effect of smoothing flat parts of the image while edge parts are strongly preserved. However, we remark that, in the case of show-through removal, TV must be applied to the optimization variable X (not to the image vector ). Concretely, R(X) applied to X is expressed as
where ε>0 is a small positive number. The rationale behind the definition of Eq (14) is as follows. Fortunately, in the definition of matrix X,KxKy+1-th column of X coincides with as mentioned in Eq (10). Therefore, in Eq (14), TV calculated from KxKy+1-th column of X can be directly used as R(X), which corresponds to applying TV on the images. We believe that this definition of R(X) is a clever idea.
Finally, by including this regularization term, our regularized counterpart of Ahmed’s BID method to solve Eq (13) can be formulated as the following optimization problem.
When the noise is contained in the degraded input images, it is necessary to change the data fidelity term from the hard constraint into the least-squares cost function, leading to the following optimization problem.
4.2.2 Optimization method
In CS, the nuclear norm minimization problem is usually solved by using iterative Soft-Thresholding (ST) algorithm, which is known to be a particular application of Proximal Gradient (PG) method. Although this method is known to converge to an exact solution, it is necessary to compute all singular values of matrix X at every iteration. The computation of all the singular values of a large matrix significantly increases computational cost. Furthermore, the speed of convergence is also very slow, because it takes a longer time for the effect of ST to be reflected to the solution. Therefore, we used SVP instead of ST to compute the proximal mapping of nuclear norm. Although using SVP makes it impossible to solve the nuclear norm minimization exactly, it improves practical performances of the proposed method significantly in terms of convergence speed and computational efficiency [17]. In our problem, we know that the rank of X is 1 in prior to the restoration. So, we calculated only one largest singular value and discarded other singular values without computing them at each iteration. In addition, Eq (16) is solved by using APG method [16], which is known to converge faster than the standard PG method. APG method with the above SVP modification in the thresholding is summarized in ALGORITHM 1. We note that Step 4 in ALGORITHM 1 corresponds to the modified SVP part. Furthermore, in order to stabilize the restoration, elements of intermediate matrix W^k^ being estimated as less than 0 or greater than Lmax was projected to the interval [0,Lmax] in Step 3 of ALGORITHM 1. This modification also improved accuracy of the restoration in addition to the SVP modification.
ALGORITHM 1: ACCELATED PROXIMAL GRADIENT METHOD
k←k+1
Calculate
5 Experimental studies
5.1 Simulation studies
First, we performed several simulation studies by using artificially generated degraded images by the show-through. All images used in simulation experiments in this paper were artificially generated by ourselves. In all the simulation studies below, the following parameter values of image degradation were used. The P.S.F. to blur the back (front) side image was a Gaussian filter with σ = 1.0 and the size of support was set to 3×3 (pixels). The value of transmittance was set to a = 0.1. Conditions in implementing the proposed method are summarized as follows. Initial values of F1 and F2 in the APG iteration were set to the degraded images G1 and G2. The size of estimated P.S.F. was set to 5×5 (pixels), which is larger than the true size of P.S.F., and initial values of P.S.F. in the APG iteration were set to
Initial values of transmittance was set to a = 0.5. The parameter values in the proposed method were set to β = 2.5 in Eq (16) and ε = 1 in Eq (14). Finally, the step-size value in Step 2 of ALGORITHM 1 was set to , and the number of iterations was 200. The hardware and software environment used in the experimental studies is summarized in Table 1. All the images and codes used in the experiments are available from “https://zenodo.org/records/11016261”. All the experiments can be reproduced.
Table 1: Hardware and software environments used in the experimental studies.
5.1.1 Experiment to confirm the effects of regularization term
First, we confirmed the effects of TV regularization term described in Section 4.2.1. Image restorations were performed with and without the TV penalty. We used two images (S1 File) shown in Fig 3(A) and 3(B) for Experiment 1 (Fig 4(A) and 4(B) for Experiment 2), which simulate the printed matters in the front side and the back side, respectively. Also, spatial resolution of these images was 256×256 (pixels). The degraded images by the show-through are shown in Fig 3(C) and 3(D) for Experiment 1 (Fig 4(C) and 4(D) for Experiment 2). The only difference between Experiment 1 and Experiment 2 lies in the original images, where there exist no real image parts having gradation in Experiment 1 and some parts consist of a real image in Experiment 2. The restored images are shown in Fig 3(E)–3(H) for Experiment 1 (Fig 4(E)–4(H) for Experiment 2), where a small part of each image is also shown at the bottom after applying gamma-correction to enhance the difference in restoration. The values of evaluation indices (PSNR and SSIM) are summarized in Table 2. In this experiment, when the TV regularization is not used, the effect of show-through still remains in the restored images, but it could be removed with almost no error values with the TV regularization, leading to improving PSNR and SSIM values significantly.
Images of Experiment 1.(a) original image (front side), (b) original image (back side), (c) input image (front side), (d) input image (back side), (e) restored image with TV (front side), (f) restored image with TV (back side), (g) restored image without TV (front side), (h) restored image without TV (back side).
Images of Experiment 2.(a) original image (front side), (b) original image (back side), (c) input image (front side), (d) input image (back side), (e) restored image with TV (front side), (f) restored image with TV (back side), (g) restored image without TV (front side), (h) restored image without TV (back side).
Table 2: Values of evaluation indices (PSNR and SSIM) in Experiment 1 and Experiment 2.
5.1.2 Effect of regularization parameters on image quality
The proposed method involves two regularization parameters (i.e. α and β). This section demonstrates the effect of α and β on quality of the restored image. The first parameter α in Eq (16) is the parameter to evaluate the magnitude of the nuclear norm. However, the proposed method with the recommended AGP + SVP optimization method does not need to adjust α. This is because it uses the singular value projection (SVP) instead of the nuclear norm reduction, which directly imposes the constraint that the rank of X is one by computing only one largest singular value and discarding other singular values. The second parameter β affects quality of the restored image. To evaluate the effect of β on the image quality, the same simulation experiments as in Experiment 1 and Experiment 2 were performed with five different values of β (i.e. β = 0.0,1.0,2.5,5.0,10.0). These experiments are named as Experiment 3 and Experiment 4, respectively. Figs 5 and 6 show zoomed small parts of the restored images for each value of β. Table 3 summarizes PSNR and SSIM indices for each value of β. Fig 7 shows changes of PSNR and SSIM indices with different β values. It is observed that increasing β value has no significant effect on large edge parts such as those in Figs 5(A), 5(B) and 6(A), but it does smooth out fine edges such as those in Fig 6(B). From Fig 7 and Table 3, it is also observed that if β is made too large, PSNR and SSIM indices become worse and β = 2.5 yields the best restoration accuracy in average for the setup of Experiments 3 and 4. Finally, we mention that the image quality was not very sensitive to β value than what we expected before these experiments.
Zoomed small parts of the restored images in Experiment 3.(a) original image (front side), (b) restored image β = 0.0 (front side), (c) restored image β = 1.0 (front side), (d) restored image β = 2.5 (front side), (e) restored image β = 5.0 (front side), (f) restored image β = 10.0 (front side), (g) original image (back side), (h) restored image β = 0.0 (front side), (i) restored image β = 1.0 (back side), (j) restored image β = 2.5 (back side), (k) restored image β = 5.0 (back side), (l) restored image β = 10.0 (back side).
Zoomed small parts of the restored images in Experiment 4.(a) original image (front side), (b) restored image β = 0.0 (front side), (c) restored image β = 1.0 (front side), (d) restored image β = 2.5 (front side), (e) restored image β = 5.0 (front side), (f) restored image β = 10.0 (front side), (g) original image (back side), (h) restored image β = 0.0 (front side), (i) restored image β = 1.0 (back side), (j) restored image β = 2.5 (back side), (k) restored image β = 5.0 (back side), (l) restored image β = 10.0 (back side).
Changes of evaluation indices (PSNR and SSIM) according to β value in Experiment 3 and Experiment 4. (a) PSNR, (b) SSIM.
Table 3: Values of evaluation indices (PSNR and SSIM) in Experiment 3 and Experiment 4.
5.1.3 Experiment to evaluate convergence
The convergence of the proposed method with the recommended APG + SVP optimization method was evaluated. The same simulation experiments as in Experiment 1 and Experiment 2 were performed with 1000 iterations of APG + SVP method. These experiments are named as Experiment 5 and Experiment 6, respectively. The changes in PSNR and SSIM indices are shown in Fig 8. From Fig 8, it is observed that both PSNR and SSIM converges at 200 iterations and the convergence is stable with no divergence or oscillating behavior. From this experiment, it can be said that the convergence of the method is guaranteed and 200 iterations are sufficient for the convergence.
Changes of evaluation indices (PSNR and SSIM) according to the iteration number in Experiment 5 and Experiment 6.(a) PSNR, (b) SSIM.
5.1.4 Experiment to confirm the effects of APG and SVP
Next, we confirmed the effects of APG and SVP introduced in Section 4.2.2. Image restorations were performed by changing the combination of APG (or PG) and SVP (or ST) to determine which iterative method converges faster. The input images in Experiment 7 and Experiment 8 were the same as those of Experiment 1 and Experiment 2, respectively. Image restorations were performed with the following four different combinations: APG + SVP (i.e. the method proposed in this paper), PG + SVP, APG + ST, and PG + ST. In this experiment, we judged that the iteration converged when PSNR indices of restored images of both the front and back sides reached to 39 (dB), and the required number of iterations up to the convergence is summarized in Table 4. The changes in PSNR by iterations are shown in Fig 9. As a result, the iteration did not converge with APG + ST and PG + ST. This non-convergence was caused due to the following reason. In APG + ST and PG + ST, the strength of rank 1 constraint imposed on the matrix X is very weak so that a very large number of iterations were necessary up to the convergence. On the other hand, APG + SVP and PG + SVP converged to 39 (dB) in PSNR. However, the number of iterations in APG + SVP was more than 16 times smaller than that of PG + SVP. From this experiment, we conclude that APG + SVP is the best method in terms of convergence speed and computational efficiency.
Changes of PSNR index according to the iteration number in Experiment 7 and Experiment 8.(a) Experiment 7, (b) Experiment 8.
Table 4: Number of required iterations for each iterative method to reach to 39 (dB) in PSNR for Experiment 7 and Experiment 8.
5.1.5 Comparison studies with other methods
To make a comparison with the existing other methods, we chose two typical and well-known methods. The first method is Sharma’s method [12], which is based on the adaptive filter. In this method, it is necessary to input a pixel value of the portion without printed matter and a threshold value to determine whether the show-through occurred or not at each pixel. In this experiment, we adjusted values of these parameter through various trial restorations, and used the same values in all experiments. The second method is NMF method [9]. Unlike Sharma’s method, this method does not need a priori input parameters concerning images to be restored, but the restoration results largely depend on initial values of the iterative method. In this experiment, we used a random initial value in all experiments. The input images in Experiment 9 and Experiment 10 were the same as those of Experiment 1 and Experiment 2, respectively. All the restored images and the corresponding error images by the three methods are shown in Fig 10 for Experiment 9 (Fig 11 for Experiment 10). The corresponding values of evaluation indices (PSNR and SSIM) are summarized in Table 5. The show-through still remains in Fig 10(G) and 10(H) in the NMF method, and Figs 10(E) and 11(F) in Sharma’s method. However, the proposed method successfully removed the show-through in any image. In Table 5, the proposed method achieved the best PSNR and SSIM values for all the images. From the PSNR and SSIM values, it is clear that the proposed method is superior to the compared other methods.
Images of Experiment 9.(a) original image (front side), (b) original image (back side), (c) restored image by proposed method (front side), (d) restored image by proposed method (back side), (e) restored image by Sharma’s method (front side), (f) restored image by Sharma’s method (back side), (g) restored image by NMF method (front side), (h) restored image by NMF method (back side).
Images of Experiment 10.(a) original image (front side), (b) original image (back side), (c) restored image by proposed method (front side), (d) restored image by proposed method (back side), (e) restored image by Sharma’s method (front side), (f) restored image by Sharma’s method (back side), (g) restored image by NMF method (front side), (h) restored image by NMF method (back side).
Table 5: Values of evaluation indices (PSNR and SSIM) in Experiment 9 and Experiment 10.
5.1.6 Evaluation of computation time
Computation times of the proposed method and the comparison methods in Experiments 7–10 were evaluated under the environment of Table 1. Table 6 summarizes the computation time for each optimization method in Experiment 7 and Experiment 8. Table 7 summarizes the computation time for each comparison method in Experiment 9 and Experiment 10. From Table 6, it is observed that APG + SVP optimization method reduces the computation time more than 14 times than PG + SVP. APG + ST and PG + ST were not evaluated because they did not converge. From Table 7, it is observed that the proposed method needs a longer computation time than other methods. This drawback needs to be overcome in future work, for example by using parallel processing.
Table 6: Computation time for Experiment 7 and Experiment 8.
Table 7: Computation time for Experiment 9 and Experiment 10.
5.2 Real image experiments
Next, we applied the proposed method, Sharma’s method, and the NMF method to images of actual papers scanned by using an electronic scanner. We call these real data experiments as Experiment 11, Experiment 12, and Experiment 13. We used two input images (S2 File) shown in Fig 12(A) and 12(B) for Experiment 11(Fig 13(A) and 13(B) for Experiment 12, Fig 14(A) and 14(B) for Experiment 13). All images used in the real image experiments were acquired by ourselves by scanning real papers having printed materials on the both sides using an electronic scanner. We note that the images used in the three experiments have different characters described below to one another. The images in Experiment 11consist of text parts both in the front and back sides. The images in Experiment 12 consist of text part in the front side and drawing part in the back side. The images in Experiment 13 consist of real images in the front side and text part in the back side. Also, spatial resolution of the input images in Experiment 11, Experiment 12, and Experiment 13 was 256×256 (pixels), respectively. In implementing the proposed method, values of the parameters and initial values of the iteration were set to the same values as in the simulation studies. All the results are summarized in Fig 12 for Experiment 11 (Fig 13 for Experiment 12, Fig 14 for Experiment 13). The NMF method fails to remove the show-trough in Fig 13(H), while it removes even the true printed matter on the front side in Figs 12(G), 14(G), and 14(H). Both Sharma’s method and the proposed method could remove the show-through well, but Figs 13(F) and 14(E) shows that Sharma’s method also removed the true printed matter in some parts. In the proposed method, such incorrect removals hardly occurred.
Images of Experiment 11.(a) input image (front side), (b) input image (back side), (c) restored image by proposed method (front side), (d) restored image by proposed method (back side), (e) restored image by Sharma’s method (front side), (f) restored image by Sharma’s method (back side), (g) restored image by NMF method (front side), (h) restored image by NMF method (back side).
Images of Experiment 12.(a) input image (front side), (b) input image (back side), (c) restored image by proposed method (front side), (d) restored image by proposed method (back side), (e) restored image by Sharma’s method (front side), (f) restored image by Sharma’s method (back side), (g) restored image by NMF method (front side), (h) restored image by NMF method (back side).
Images of Experiment 13.(a) input image (front side), (b) input image (back side), (c) restored image by proposed method (front side), (d) restored image by proposed method (back side), (e) restored image by Sharma’s method (front side), (f) restored image by Sharma’s method (back side), (g) restored image by NMF method (front side), (h) restored image by NMF method (back side).
5.3 Discussion
From the simulation results in this section, the following six facts about the proposed method can be said. First, the show-through removal by the proposed method improves accuracy by using the TV regularization term. Second, the effect of the choice of β in Eq (16) affects the accuracy of the restored image. Third, the proposed method has good convergence. Forth, the use of APG + SVP as the iterative solution method is able to speed up convergence, improve computational efficiency, and also improve accuracy. Fifth, the proposed method provides accurate and stable restoration results with the simpler parameter adjustment compared with Sharma’s method and the NMF method. The NMF method is a simple method that does not need the parameter adjustment, but it is not practical because initial values of the iteration change the solution significantly. On the other hand, Sharma’s method is able to remove the show-through well if the two parameters, i.e. the pixel value of portion without the printed matter and the threshold value to determine whether the show-through occurred or not at each pixel, are set to appropriate values. However, the threshold value needs to be changed according to the density of printed matter, and the pixel value of portion without the printed matter also needs to be changed according to the type of paper. On the other hand, the parameter adjustment in the proposed method is easier allowing an accurate and stable show-through removal for various different images under different conditions. Sixth, the proposed method has a longer computation time than other methods.
6 Conclusion
In this paper, we proposed a new method of the show-through removal which takes the blurring into account by applying Ahmed’s BID method. From the results of experimental studies, it became clear that the proposed method is able to achieve accurate and stable restorations with the relatively simple parameter adjustments. However, the proposed method still has a limitation that values of the transmittance and the P.S.F. need to be same at every position on the paper. In addition, these values need to be same in the front side and in the back side. In future work, it is necessary to overcome this limitation by developing a space-variant degradation model of show-through and a corresponding restoration method, which better match to real situations. Finally, the most important point to be improved in this method is the reduction of computation time. Reducing computational complexity and increasing convergence speed are also desired.
Supporting information
S1 FileOriginal images used in the simulation experiments.(ZIP)
S2 FileImages used in the real image experiments.(ZIP)
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Sauvola J. and Pietikainen M.," Adaptive document image binarization," Pattern Recognit., 33(2), 225–236 (2000).
- 2Kim S.J., Jang M.S., and Park G.T.," Image contrast enhancement based on segmentation and tone curve control for LCD TV," Digest of Technical Papers International Conference on Consumer Electronics, 1–2 (2009).
- 3Tonazzini A., Salerno E., and Bedini L.," Fast correction of bleed-through distortion in grayscale documents by a blind source separation technique," Int.J.Doc.Anal Recognit., 10(1), 17–25 (2007).
- 4Almeida M.S.C. and Almeida L.B.," Nonlinear separation of show-through image mixtures using a physical model trained with ICA," Signal Process., 92(4), 872–884 (2012).
- 5Merrikh-Bayat F., Babaie-Zadeh M., and Jutten C.," Using non-negative matrix factorization for removing show-through," Latent Variable Analysis and Signal Separation. LVA/ICA 2010, Lecture Notes in Computer Science (Vigneron V., Zarzoso V., Moreau E., Gribonval R., and, Vincent E., eds), Vol.6365, 482–489 (2010).
- 6Sharma G.," Show-through cancellation in scans of duplex printed documents," IEEE Trans.Image Process., 10(5), 736–754 (2001). doi: 10.1109/83.918567 18249664 · doi ↗ · pubmed ↗
- 7Nakahara T. et al.," Deep convolutional neural networks for manga show-through cancellation," IEICE Ttans.Inf. & Syst., E 101-D(11), 2844–2848 (2018). doi: 10.1587/transinf.2018 EDL 8051 · doi ↗
- 8Berahmand K., Daneshfa F., Salehi E.S., and Li Y.,”Autoencoders and their applications in machine learning: a survey," Artif Intell Rev., 57(2), 28 (2024).