Hybrid Iterating-Averaging Low Photon Budget Gabor Holographic Microscopy

TL;DR

This paper introduces a new algorithm for low-light microscopy that improves image quality and reduces cell damage during long-term observations.

Contribution

The novel IGA algorithm combines iterative phase retrieval with frame averaging to suppress noise and twin image artifacts in low-photon holographic imaging.

Findings

IGA outperforms conventional methods in reconstructing phase images under high-noise conditions.

Experimental validation confirmed IGA's efficacy in high-speed imaging of dynamic biological samples.

IGA works well for optically thin samples with low signal-to-noise ratios.

Abstract

Achieving high-contrast,
label-free imaging with minimal impact
on live cell culture behavior remains a primary challenge in quantitative
phase imaging (QPI). By enabling imaging under low illumination intensities
(low photon budget, LPB), it is possible to minimize cell photostimulation,
phototoxicity, and photodamage while supporting long-term and high-speed
observations. However, LPB imaging introduces significant difficulties
in QPI due to high levels of camera shot noise and quantification
noise. Digital in-line holographic microscopy (DIHM) is a QPI technique
known for its robustness against LPB data. However, simultaneous minimization
of shot noise and inherent in DIHM twin image perturbation remains
a critical challenge. In this study, we present the iterative Gabor
averaging (IGA) algorithm, a novel approach that integrates iterative
phase retrieval with frame averaging to effectively suppress both
twin image disturbance and shot noise in multiframe DIHM. The IGA
algorithm achieves this by leveraging an iterative process that reconstructs
high-fidelity phase images while selectively averaging camera shot
noise across frames. Our simulations demonstrate that IGA consistently
outperforms conventional methods, achieving superior reconstruction
accuracy, particularly under high-noise conditions. Experimental validations
involving high-speed imaging of dynamic sperm cells and a static phase
test target measurement under low illumination further confirmed IGA’s
efficacy. The algorithm also proved successful for optically thin
samples, which often yield low signal-to-noise holograms even at high
photon budgets. These advancements make IGA a powerful tool for photostimulation-free,
high-speed imaging of dynamic biological samples and enhance the ability
to image samples with extremely low optical thickness, potentially
transforming biomedical and environmental applications in low-light
settings.

Figures

Scheme of (a) lens-based and (b) lensless DIHM systems.

(a) Simulation results of GS twin image minimization for shot noise-free and shot noise-spoiled data (illustrated for 1 and 100 iterations performed). (b) Reconstruction RMS error plots computed in a function of the number of iterations (from 1 to 100). There were simulated holograms (collected with different defocus) of a purely amplitude object with amplitude values ranging from 0 to 1 [a.u.]. Noisy data holograms were spoiled with a Gaussian noise of 0.05 standard deviation.

(a) Simulated purely amplitude object and (c1) object spoiled with shot noise, (c2) twin image noise and (c3) both noise types. (b) Fourier transform (horizontal frequencies) of noise error maps (differences between simulated object and the noise spoiled image) from (c1–c3), showing the frequency response of different noises. (d) Proposed IGA algorithm processing path for reconstruction of the object shown in (a) from two in-line holograms collected with different defocus distance and spoiled with shot noise (shown in top-left corner of (d)). In order to make the algorithm scheme clearer only the amplitude part of reconstructed optical fields is shown for GS, GA and IGA methods.

(a) Simulated purely phase object. (b) Reconstructed phase SSIM values for different algorithms and different input data noise levels. (c) Exemplifying reconstructions (zoomed-in letter Q from the investigated object) for shot noise free and shot noise spoiled data. (d) Cross-section for shot noise spoiled data shown in (c).

(a) Input in-line holograms recorded under high (HPB) and low (LBP1, LBP2) photon budget conditions, and (b) their histograms. (c) GS, (d) GA, and (e) proposed IGA reconstruction results. Left, central and right columns correspond to HPB, LPB1, and LPB2 data sets throughout the entire figure. L—laser irradiance measured in camera plane, M—hologram mean value (brightness), ΔM—hologram standard deviation (contrast), S—standard deviation of object-free area marked with yellow rectangle given in [rad] units. SSIM results are given for comparing reconstructed object phase (without background area) with the GS reconstructed object for the HPB data.

Phase images of a dynamic live human sperm sample reconstructed with investigated algorithms (GS, GA, and IGA). Results are shown for first frames of HPB and LPB data sets. Full time-lapse reconstructions of each imaging case are shown in Supporting Videos 1–6.

(a) Phase reconstructions of optically thin resolution phase test target. S values are given in [10–2 rad] units. (b) Cross-section of the vertical elements from group M compared to the reference height measurement from white light interferometer.

Tables

Table 1. Reconstruction Time of Different Algorithms for Input Data Composed of Three Multiplexed Holograms (1000 pixels × 1000 pixels)a

	GS	GA	IGA
reconstruction time [s]	1.22	0.17	1.40

Full text

Introduction

Imaging live biosamples, such as cell cultures, poses a significant challenge in optical microscopy due to their inherent transparency, which leads to generally very low contrast. Quantitative phase imaging (QPI) techniques have emerged as a popular solution for visualizing transparent objects with high contrast, independent of their absorption properties.^1−4^ QPI methods are based on measuring the phase delay between the light wave that passes through the sample and the light wave that passes through its surrounding medium. As long as the optical thickness of the sample and the surrounding medium differs, QPI methods can provide high-contrast images even for completely transparent samples. Additionally, QPI reconstruction yields quantitative information about the sample’s optical thickness.^5^

Among the various QPI methods, digital in-line holographic microscopy (DIHM) is particularly noteworthy.^6^ DIHM setups distinguish themselves from other QPI techniques due to their simplicity and cost-effectiveness. For example, DIHM can be implemented in classical brightfield microscopes with added coherent illumination module.^7,8^ But probably the biggest advantages come in lensless microscopy configuration (lensless DIHM),^9,10^ which, aside from being extremely simple, size-optimized and cost-effective (as it eliminates the need for microscope objectives and embodiments), offers exceptionally large fields of view (easily over 100 mm^2^) with reasonable spatial resolution (typically around 1–2 μm)—a so-called high space-bandwidth-product imaging.^11^ By eliminating microscope objectives, lensless phase microscopes also avoid optical aberrations, depth of field, and working distance limitations.

DIHM belongs to a broader category of computational microscopy techniques^12,13^ where data captured by the camera requires appropriate processing to reconstruct physically meaningful information. Although DIHM reconstruction ideally requires only a single image (in-line hologram), single-frame reconstructions are plagued by the twin image effect^9,10^—a disturbance numerical phenomenon resulting from lack of phase information at the camera plane. Minimizing this effect—and thus improving QPI results—from a single hologram is an ill-posed problem, which can be partially addressed through regularization^14,15^ or deep learning approaches.^16,17^ More robust methods for twin image minimization involve capturing multiple images under different imaging conditions (e.g., different camera positions along the optical axis^18,19^ or different illumination wavelengths^20−22^). Multiframe reconstruction is a well-posed problem that can efficiently converge to the true solution. However, its main drawback is the need to collect multiple images, which is usually done sequentially, reducing temporal resolution and increasing hardware burden.

Another significant challenge in imaging live cell cultures is to avoid the photostimulation^23^ and in more severe cases phototoxicity^24,25^ and photodamage^26,27^ of the cells due to the influence of illumination light. This influence can be mitigated by reducing the illumination radiation dose, leading to low photon budget (LPB) imaging conditions characterized by low illumination power relative to the measurement time (camera exposure time). LPB conditions can also arise when imaging samples in high speed scenario^28^ or using exotic wavelengths that greatly fall outside the detector’s optimal quantum efficiency range.^29,30^

LPB imaging conditions in QPI^31−35^ are challenging due to two primary factors: (1) low signal-to-noise ratio (SNR), resulting in images marred by camera shot noise, and (2) low signal intensity, leading to quantification noise (when the signal is sampled by a low number of camera gray levels). Interestingly, as shown by our previous results,^36^ DIHM demonstrates great potential in LPB imaging using single-frame setups due to its reduced sensitivity to quantification noise compared to other QPI systems. This capability allowed for high-accuracy DIHM reconstructions even from images collected with only a few camera’s gray levels. Nonetheless, the persistent challenges include the presence of camera shot noise and twin image perturbation, which are very difficult to minimize simultaneously from a single acquired image (although training deep neural network with noisy data^31−34,37^ or employing appropriate regularizations^15,38,39^ may potentially allow for enhanced reconstructions).

Intuitively, the drawbacks of single-frame DIHM methods in LPB imaging should be mitigated by multiframe DIHM techniques. As previously stated, multiframe iterative DIHM algorithms are more robust against twin image disturbance than single-frame methods. Additionally, acquiring multiple images can also aid in averaging shot noise. However, the conventional iterative QPI methods are generally known for their low accuracy when working with noisy data,^40−42^ and as shown by our novel results presented in this paper, this is also the case in DIHM.

In this work, we propose a novel, open-source,^43^ multiframe DIHM reconstruction framework specifically designed to simultaneously minimize shot noise and twin image disturbance in LPB conditions. We evaluate the robustness of the proposed method through numerical simulations, demonstrating, for the first time to the best of our knowledge, the ability to effectively reduce both types of noise without compromising image resolution. The numerical results are validated by experimental LPB imaging outcomes of (1) a static two-photon polymerization (TPP) printed phase object measured in a lensless DIHM setup and (2) a dynamic sperm cell sample imaged in a high-speed brightfield microscope DIHM. Furthermore, we show that the proposed algorithm can be successfully applied to measure extremely low phase change samples by imaging a phase test target with a thickness of only 16 nm (0.08 rad phase thickness)—a very challenging case even under high photon budget (HPB) imaging conditions.

Methods

DIHM and Lensless DIHM Systems

Figure 1a illustrates the scheme of the DIHM system used in this study that, essentially, was the same one as presented in references.^7,8^ The system consisted of a regular upright microscope embodiment (Olympus BX-60 with UMPlanFl 20X 0.46NA objective) where a fiber coupled diode laser was externally inserted to provide coherent illumination in the system. The multiwavelength laser source (Blue Sky Research SpectraTec 4 STEC4 405/450/532/635 nm) provided RGB (450/532/635 nm) illumination and a RGB camera (Ximea USB3MQ042CG-CM, CMOS sensor type, 2048 × 2048 pixels, 5.5 μm pixel pitch, 90 fps) was used to record the RGB holograms in a single snapshot. As in previous setups,^7,8^ the sample was moved away from the microscope lens so that the image plane was located before the digital sensor plane. Under these conditions, the camera recorded three color-coded in-line holograms with free-space propagation defocus each one of them from the image plane as in a classical Gabor configuration. LPB imaging conditions were introduced by lowering the camera exposure time, an extremely important regime in high-speed imaging scenarios.

Scheme of (a) lens-based and (b) lensless DIHM systems.

Figure 1b depicts the scheme of the lensless DIHM system employed. The laser used in this system (CNI Lasers MGL-FN-561-20 mW, λ = 561 nm) emitted light with linear polarization. To facilitate easy control of the illumination power, a linear polarizer was positioned directly after the laser. By rotating the polarizer, the illumination intensity could be easily adjusted, ranging from zero to the maximum laser power. The light was then coupled into a fiber (Thorlabs P1-460B-FC-1), with the fiber end aligned on-axis with the camera (ALVIUM Camera 1800 U-2050 m mono Bareboard, pixel size 2.4 μm × 2.4 μm, 5496 pixels × 3672 pixels), at approximately 20 cm from the camera plane. The camera was mounted on a translational stage (Thorlabs KMTS25E/M), allowing for precise control over the distance between camera and sample planes. Measured sample was placed around 3–5 mm away from the camera. Since the camera-sample distance was much shorter than the distance between camera and fiber end, the system’s geometrical magnification was close to 1.

Despite the significant differences, both systems are based on the same physical principles (Gabor in-line holography), allowing for similar processing paths to reconstruct the data acquired. In both systems, the sample was illuminated with a quasi-monochromatic wave illumination. The sample’s complex optical field was then propagated in free space to the camera (in the lensless DIHM system) or to the objective focal plane and subsequently imaged onto the camera (in the DIHM system). The camera captures the amplitude part of the free-space propagated (defocused) optical field (in-line hologram), which can be then numerically backpropagated^44,45^ to the sample/image plane to retrieve its complex (amplitude and phase) distribution—this is known as the Gabor holography reconstruction scheme (GHR).^46^

Due to the lack of the phase information in the recording process performed at the camera plane, GHR is compromised by the twin image presence. As discussed in the Introduction, this effect can be minimized via an iterative approach introducing data multiplexity—collecting several different holograms using varied illumination wavelengths or defocus distances. In the used DIHM system, data multiplexity was achieved by illuminating the sample with three different RGB wavelengths, thereby producing three different holograms simultaneously, each captured by a different color channel of the RGB camera. In the lensless DIHM system, data multiplexity was achieved by moving the camera along the optical axis and collecting sequentially three holograms with different defocus (sample-to-camera) distances.

Used systems differ in their imaging capabilities. The DIHM system’s imaging resolution is diffraction-limited (to 0.8 μm), with a field of view constrained by the camera sensor dimensions and the optical magnification (to 0.39 mm × 0.39 mm), and its temporal resolution is limited by the camera frame rate (90 fps). The lensless DIHM system’s resolution is limited by the effective camera pixel size (2.4 μm—in this case more demanding than diffraction limited resolution^45,47^), its field of view is limited solely by the camera sensor dimensions (13.19 mm × 8.81 mm) due to ≅1 magnification, and its temporal resolution is restricted by the need to scan the camera along the optical axis and acquire multiple images (a single unoptimized measurement of three scanned heights took approximately 2 s).

Proposed Iterative Gabor

Averaging Algorithm

Traditionally, twin image disturbance can be minimized from multiplexed holograms using various iterative algorithms, such as Gerchberg-Saxton (GS),^48^ hybrid input-output,^49^ or conjugate gradient.^50^ These methods rely on finding the complex optical field, which after numerical backpropagation to different hologram planes, will match those holograms. This approach converges to the true solution if the acquired holograms accurately represent the amplitude part of the optical field. However, this assumption does not hold in the presence of noisy holograms, which also contain information unrelated to the reconstructed optical field (i.e., camera shot noise).

Figure 2 presents a simple simulation showing the performance of iterative GS algorithm in terms of twin image minimization for noisy and noise-free DIHM data. In this simulation, the GS algorithm was provided with two multiplexed holograms (collected with different defocus distances) of a purely amplitude object and performed for 100 iterations with exemplifying results shown in Figure 2a. For shot noise-free data, the GS converged to true solution, efficiently minimizing the twin image contribution, see plot in Figure 2b. However, for shot noise-spoiled data, the reconstruction root-mean-square (RMS) error increases with the number of iterations (Figure 2b). Closer analysis of the reconstruction results reveals that GS managed to minimize the twin image disturbance at the cost of shot noise multiplication, which is an extremely alarming effect in LPB imaging. This effect results from the GS nature itself, which tries to find a solution that matches the uncorrelated noise in the multiframe input data. To the best of our knowledge, this property was not shown in the DIHM before, although it is consistent with observations made for iterative phase retrieval methods in different QPI systems.^40−42^

(a) Simulation results of GS twin image minimization for shot noise-free and shot noise-spoiled data (illustrated for 1 and 100 iterations performed). (b) Reconstruction RMS error plots computed in a function of the number of iterations (from 1 to 100). There were simulated holograms (collected with different defocus) of a purely amplitude object with amplitude values ranging from 0 to 1 [a.u.]. Noisy data holograms were spoiled with a Gaussian noise of 0.05 standard deviation.

When having multiple images of the same object, which differ only by the shot noise (of similar level in each image), the most straightforward and natural way to minimize the noise is to average the acquired images.^51^ Shot noise can also be minimized in single-frame scenarios^52^ using methods such as spatial filtering,^53^ iterative regularization,^38^ frequency filtering^54^ or deep learning.^55^ However, for successful denoising, single-frame methods: (1) require some “a priori” knowledge about the measured object and/or noise characteristics, (2) may incur in high computational costs, and (3) may reduce spatial resolution (blurring the image). Single-frame denoising methods can also be applied in multiframe scenarios, either by first averaging the acquired images and then performing single-frame denoising or by starting with single-frame methods applied to each image and then averaging the results.

In the case of multiframe shot noise minimization of multiplexed DIHM data, frame averaging cannot be directly applied to the collected images, as the input DIHM holograms contain different object information (acquired with different defocus/illumination wavelengths). However, the averaging can be applied to the reconstructed holograms (numerically backpropagated to the object plane)—we denote this operation as Gabor averaging (GA) method. Each reconstructed hologram contains the same in-focus object information and differs by the presence of shot noise and partially by twin image disturbance (especially higher spatial frequency twin image information). Therefore, averaging them allows for not only shot noise reduction but also partial twin image presence reduction, though not to the accuracy of iterative DIHM algorithms.

Depending on the shot noise level, different multiframe reconstruction approaches may be more successful. For low shot noise levels, the twin image issue is the main source of noise, favoring iterative DIHM methods. For strong shot noise, its influence is dominant, favoring the GA approach. However, in mixed scenarios where both noise types are similarly significant, neither method will provide accurate denoising results. This is precisely the gap in the state-of-the-art we address in this contribution.

Here, we present a novel algorithm called iterative Gabor averaging (IGA), which combines the advantages of iterative and GA approaches to provide a multiframe LPB DIHM reconstruction with both noise types minimized. To design an accurate algorithm, the properties of both noise types should be analyzed. Figure 3a presents a numerically simulated amplitude-type object, and Figure 3c1–c3 shows its reconstruction spoiled by shot noise, twin image disturbance, and both noise types simultaneously. As observed, the noise types are visually distinct: shot noise is uniformly distributed across the entire reconstructed image, while twin image disturbance manifests as a diffraction pattern overlaying the measured object. The difference between the noise types can be distinguished in the frequency domain. Figure 3b presents a plot of the horizontal spatial frequencies of the noise error maps (calculated as the simulated object minus the noise-spoiled reconstruction). Shot noise is uniformly distributed across different spatial frequencies, while twin image noise is primarily concentrated in low spatial frequencies.

(a) Simulated purely amplitude object and (c1) object spoiled with shot noise, (c2) twin image noise and (c3) both noise types. (b) Fourier transform (horizontal frequencies) of noise error maps (differences between simulated object and the noise spoiled image) from (c1–c3), showing the frequency response of different noises. (d) Proposed IGA algorithm processing path for reconstruction of the object shown in (a) from two in-line holograms collected with different defocus distance and spoiled with shot noise (shown in top-left corner of (d)). In order to make the algorithm scheme clearer only the amplitude part of reconstructed optical fields is shown for GS, GA and IGA methods.

The proposed IGA method leverages the above-mentioned noise properties by combining iterative GS phase retrieval reconstruction for low-frequency information (where twin image disturbance is dominant) with GA reconstruction for high-frequency information (where shot noise is dominant). The IGA processing path is presented schematically in Figure 3d. The algorithm takes as input several (at least two) in-line holograms collected with different defocus distances/illumination wavelengths, which are then processed by two algorithm paths independently. In the first patch, each input hologram is low-pass filtered with a Gaussian kernel of user-chosen standard deviation (σ). The low-pass filtered holograms are then provided to the GS phase retrieval algorithm, which, thanks to the shot noise minimization via low-pass filtering, retrieves the twin image-free (but blurred) reconstruction without significant noise amplification. Simultaneously, in the second path, the GA method is applied to obtain shot noise averaged reconstruction from the input data. This reconstruction is then high-pass filtered with the same Gaussian kernel (same σ) as used for low-pass filtering in the first path. Finally, the results from both paths are combined by summation giving twin image minimized, shot noise averaged reconstruction without information blur (see Supporting Document 1 Sections 1–4 and the released open-source codes^43^ for the implementation details).

The proposed algorithm is based on the concept of combining high-pass and low-pass image filtering, which is fundamental in many image processing applications.^56−58^ Notably, the proposed method is relatively simple, resulting in low computational costs (insignificantly larger than the combined time of the GS and GA methods, see Table 1). The simplicity of the method also enhances its robustness, as it does not involve many steps that could introduce algorithmic errors. The algorithm’s outcome depends on two user-defined parameters: (1) the number of iterations of the GS algorithm (T), and (2) the Gaussian kernel standard deviation σ. Increasing T enhances twin image minimization accuracy but also extends reconstruction time. The σ value balances the influence between the GA and GS algorithm components—the larger the σ, the stronger the shot noise minimization and the weaker the twin image minimization. In this study we applied σ parameter equal 2 for all presented results, following the analysis performed in Supporting Document 1 Section 5. However, it is to be highlighted that in the mentioned analysis, a relatively wide range of σ values (between 1 and 4) produced similarly good results. This indicates that the σ value is a fine-tuning type of parameter rather than a critical one that must be set carefully to enable high accuracy reconstruction.

Table 1: Reconstruction Time of Different Algorithms for Input Data Composed of Three Multiplexed Holograms (1000 pixels × 1000 pixels)a

The proposed novel open-ended numerical framework for LPB holographic denoising can be easily modified while maintaining its core novel idea of hybrid GA and iterative DIHM processing. For example, the GS algorithm can be replaced with a different iterative phase retrieval method,^49,50^ or the Gaussian kernel high-pass/low-pass filtering can be substituted with another filtering method.^54^ Especially, algorithm’s performance could be further improved by adjusting the high-pass/low-pass filtration to the system shot noise characteristics^59,60^ (also resulting in algorithm automatization by eliminating the need to manually set σ parameter).^1^

Results

Simulations

To numerically evaluate the proposed algorithm, we simulated holograms acquired using the lensless DIHM system under varying shot noise levels. In the simulated system, three holograms were acquired at different camera-sample distances (2, 3, and 4 mm). The camera pixel size was set to 2 μm, and a perfectly coherent light source of 500 nm wavelength was simulated. The measured object was modeled as a purely phase object (our lab’s logo) with phase values ranging from −0.4 to 0.6 rad, as shown in Figure 4a.

(a) Simulated purely phase object. (b) Reconstructed phase SSIM values for different algorithms and different input data noise levels. (c) Exemplifying reconstructions (zoomed-in letter Q from the investigated object) for shot noise free and shot noise spoiled data. (d) Cross-section for shot noise spoiled data shown in (c).

This configuration produced in-line holograms with an average intensity value (M) of 1 au and contrast (ΔM, defined as the hologram standard deviation) of 0.07 au Camera shot noise was simulated as Poisson distribution noise with standard deviation (std) varying from 0 to 0.2 [a.u.], resulting in a minimal hologram SNR equal 0.36 (calculated as noise-free hologram ΔM over shot noise std). The simulated holograms were reconstructed using the GS, GA, and IGA methods. The GS and IGA methods were performed with T = 100 iterations to ensure good convergence of the final result. Additionally, classical GHR phase retrieval was performed using single-frame data (holograms collected at a single camera-sample distance of 2 mm).

Figure 4b shows the plot of phase structural similarity index measure (SSIM)^61^ in relation to the hologram SNR. As observed, for low shot noise levels, the GS and IGA methods achieved similarly good results with higher SSIM than straightforward GA and GHR. For higher shot noise levels, the GS performance decreased significantly compared to the other investigated algorithms—we showcase this disadvantage of GS for the first time, to the best of our knowledge, and link it to detrimental “convergence” to dominant noise (results presented in Figure 2). The proposed IGA method performed similarly well to GS in high SNR cases and similarly well to GA in low SNR cases, demonstrating that it feasibly merges the advantages of both methods without being affected by their drawbacks.

Figure 4c presents the exemplifying reconstructions zoomed-in letter Q of the simulated object achieved by the investigated methods for the noise-free and noisy cases, while Figure 4d shows a cross-section through the central part of the reconstructed letter Q for the noisy data shown in Figure 4c. It can be observed that GHR and GA reconstructions are spoiled by twin image disturbances, which manifest mainly as false positive information in the center of the letter Q. The GS method minimizes the twin image almost perfectly, however, for noisy data, the shot noise is increased. The IGA outperforms the GA method in terms of twin image minimization, achieving results nearly as good as GS for shot noise-free cases. However, unlike GS, it also performs well for shot noise spoiled data, achieving similar shot noise levels to GA.

Experiments

Figure 5 presents the evaluation of LPB QPI measurements in a lensless DIHM system. In the performed experiment, the measured object was a custom-made phase test target printed using the TPP technique (see Supporting Document 1 Section 6 for details). Three data sets (HPB, LPB1, and LPB2) were collected, each containing three in-line holograms recorded at different camera-sample distances (3, 4, and 5 mm). The HPB data set was collected with laser illumination irradiance (L) of 2.71 μW/cm^2^ (measured at the sample plane with Thorlabs S120C photodiode power sensor). For the LPB1 and LPB2 data sets, L was reduced to 0.21 and 0.11 μW/cm^2^, respectively. The camera exposure time was set to 10 ms for each data set. This combination of laser irradiance and camera exposure time resulted in the acquisition of reference “bright” holograms in the HPB data (utilizing most of the 8 bits camera’s dynamic range) and “dark” holograms in the LPB data (utilizing only the few first camera gray levels). The brightness and contrast of the measured holograms were quantified as their mean value (M) and their standard deviation (ΔM), respectively. The M and ΔM values (given in camera gray level (cgl) units), along with exemplifying collected holograms and their histograms, are shown in Figure 5a,b, respectively.

(a) Input in-line holograms recorded under high (HPB) and low (LBP1, LBP2) photon budget conditions, and (b) their histograms. (c) GS, (d) GA, and (e) proposed IGA reconstruction results. Left, central and right columns correspond to HPB, LPB1, and LPB2 data sets throughout the entire figure. L—laser irradiance measured in camera plane, M—hologram mean value (brightness), ΔM—hologram standard deviation (contrast), S—standard deviation of object-free area marked with yellow rectangle given in [rad] units. SSIM results are given for comparing reconstructed object phase (without background area) with the GS reconstructed object for the HPB data.

The acquired data sets were then reconstructed using the GS, GA, and IGA algorithms. The obtained phase results are shown in Figure 5c–e, respectively. These results were also compared quantitatively by calculating the std of the object-free area (S) marked with a yellow rectangle in Figure 5 and by comparing the reconstructed object phases (without background area) with reference object phase (one reconstructed for HPB data with GS method) with the use of SSIM. The results achieved by the GA algorithm were again spoiled by twin image disturbance, visible as a diffraction pattern around the measured object. The GS and IGA methods reduced most of the twin image noise, resulting in a significantly more uniform background around the object in the case of HPB data, as quantitatively confirmed by the S measure. The GS results for LPB data were spoiled by shot noise, whereas the GA and IGA methods seemed to be much more robust to its negative influence.

Interestingly, all three methods managed to reconstruct the LPB1 data set with only slightly worse accuracy (S measurement) than the HPB data set, despite the fact that the LPB1 data set was collected with over 12 times lower illumination power than the HPB data set. This once again indicates the ability of DIHM to work effectively with LPB data.^36^

One of the potential applications where LPB imaging conditions are critical is high-speed imaging of dynamic biospecimen. In such cases, the low illumination radiation dose required to avoid stimulating or damaging the sample cannot be compensated by increasing the camera exposure time, as this would reduce both the measurement’s temporal (as a direct consequence of the frame rate enlargement) and lateral (because higher exposure times will allow sample’s movement thus averaging final acquired image and presenting blurred reconstructions) resolutions.

Typically, multiframe DIHM systems are configured to collect the holograms needed for a single-instance reconstruction sequentially (as in lensless DIHM in Figure 1b), which limits the system’s temporal resolution. However, in some configurations, the required input holograms can be collected simultaneously (e.g., using wavelength multiplexing, as in DIHM in Figure 1a), allowing for imaging at the camera’s frame rate at the expenses of defining a higher effective pixel size.

Figure 6 presents the results of high-speed imaging of human spermatozoa (see Supporting Document 1 Section 7 for sample preparation and measurement details). In this experiment, we employed a wavelength multiplexing lens-based DIHM system (Figure 1a), collecting two sets of time-series data (HPB: M ± ΔM = 74.05 ± 22.57 [cgl] and LPB: M ± ΔM = 4.08 ± 1.81 [cgl]), acquired with different camera exposure times (HPB—4 ms and LPB—0.3 ms) but with the same illumination laser power. Each data set comprised 300 RGB images collected at 90 fps. Each of the three image channels was treated as a single hologram collected with a different illumination wavelength. These three holograms were then processed using the GS, GA, and IGA methods to retrieve the phase image in each frame (since the object phase values are proportional to the illumination wavelengths, the employed algorithms were modified to account for this factor—see their implementation details in Supporting Document 1 Sections 1–4). The results in Figure 6 are shown for the first of the collected frames. The time-lapse reconstructions of all frames for both data sets and each investigated algorithm are shown in Supporting Videos 1–6.

Phase images of a dynamic live human sperm sample reconstructed with investigated algorithms (GS, GA, and IGA). Results are shown for first frames of HPB and LPB data sets. Full time-lapse reconstructions of each imaging case are shown in Supporting Videos 1–6.

The obtained reconstruction results confirm the ability of the IGA method to minimize twin image contribution with a quality similar to the GS method while maintaining the GA method’s ability to average high-frequency shot noise from multiple acquired images. However, similarly to GS, IGA reconstruction is also spoiled by low-frequency background fluctuations, which are not filtered out in the low-pass filtering step of IGA algorithm. Nevertheless, it is noteworthy that the IGA method minimizes shot noise by averaging multiple frames (which are necessary to collect in multiframe DIHM anyway), thus avoiding any loss of spatial resolution. This property is confirmed by the experiment, where the spermatozoid tails (approximately 0.4–0.5 μm width^62^—below the objective’s diffraction-limited resolution of 0.8 μm) are visible with the same quality and sharpness in the IGA reconstruction as in the GS method. Moreover, in the GS LPB reconstruction, the sperm tail appears with worse contrast due to shot noise amplification. The IGA method averages the shot noise without blurring the tail, demonstrating its effectiveness in high-speed, low-photon imaging scenarios.

The proposed IGA algorithm is not only limited to LPB imaging. It can be in general applied to any multiframe DIHM system with low SNR data. This scenario may arise when imaging samples with extremely low optical thickness. Such objects scatter light minimally, resulting in holograms with low signal values (and therefore low SNR) even in HPB illumination regimes.

Figure 7 presents the QPI results of the lensless DIHM (Figure 1b) of an optically thin phase resolution test target (custom-made—Lyncée Tec, Borofloat 33 glass, 1.47 refractive index). The test’s physical thickness, given by the manufacturer is 16 ± 3 nm, corresponding to a phase change of 8.47 ± 1.59 [10^–2^ rad] for the employed wavelength (561 nm). In the performed system, we collected three in-line holograms with different camera-sample distances (4.5, 6.5, and 8.5 mm). The holograms were acquired with camera exposure times adjusted to the illumination intensity, resulting in a M value of 50.95 cgl—similar to HPB data in previous experiments. However, due to the weak scattering of light by the optically thin sample, the ΔM value was only 4.22 cgl, which is more comparable to LPB than HPB data from previous experiments.

(a) Phase reconstructions of optically thin resolution phase test target. S values are given in [10–2 rad] units. (b) Cross-section of the vertical elements from group M compared to the reference height measurement from white light interferometer.

The obtained results corroborate the usefulness of the proposed algorithm for DIHM reconstruction of low SNR data. The GA method suffers from twin image disturbances, visible mainly as a “halo” around larger test elements and as a diffraction pattern around optically thicker dust particles contaminating the sample. The GS algorithm successfully minimizes the twin image contribution; however, it amplifies the shot noise, which reduces the spatial resolution (compare zoomed-in elements from group Q in Figure 7a). The IGA method minimizes both noise types, as confirmed both qualitatively and quantitatively by calculating the S measurement from the object-free area marked with a yellow rectangle in Figure 7a. The height of the phase test elements estimated by the proposed method closely matches the reference height provided by the manufacturer (see the cross-section in Figure 7b). This demonstrates the IGA algorithm’s effectiveness in accurately reconstructing low SNR holographic data, maintaining spatial resolution while minimizing both shot noise and twin image disturbances.

Conclusions

and Discussion

Low photon budget (LPB) imaging is crucial in various applications, particularly in photostimulation-free living cell imaging, high-speed imaging of dynamic biospecimens and in scenarios where maintaining the natural environment of live cell cultures is essential. The primary challenge in LPB imaging is to obtain high-quality images despite the reduced illumination, which leads to low SNR data.

As shown by the previous works,^36^ DIHM technique shows promising prospects in QPI under LPB conditions. However, the conventional DIHM reconstruction algorithms struggle in simultaneous minimizing of camera shot noise (present due to LPB) and twin image effect (inherent in DIHM). In this work, we addressed these challenges by proposing a novel IGA algorithm tailored for multiframe DIHM under LPB conditions. The IGA algorithm combines the strengths of iterative phase retrieval methods and frame averaging to effectively minimize both twin image contribution and shot noise, what was validated numerically and experimentally. Additionally, the IGA algorithm proved effectiveness in reconstructing other kinds of low SNR DIHM data—HPB images of objects with extremely low optical thickness.

The potential challenges for future development of the IGA method might consider addressing the problem of automatic σ parameter adjustment to the input data, which currently requires manual optimization to balance the algorithm’s contributions from GA and GS components. By implementing an adaptive approach, the σ parameter could be dynamically tailored to the noise characteristics and contrast levels of each specific data set, enabling IGA to self-optimize for various sample types and imaging conditions. Additionally, further noise reduction could be achieved through the extra integration of single-frame spatial filtering techniques. While the present IGA framework relies on averaging camera noise across multiple frames to enhance the SNR, the application of advanced spatial filters could help isolate and reduce residual noise at the individual frame level. This dual approach (Gabor averaging combined with spatial filtering) could further increase the algorithm’s robustness in high-noise, low-illumination settings, making it even more effective for imaging low-contrast, dynamic biological samples. We plan to address this interesting opportunity in future works.

In summary, the proposed IGA algorithm offers a novel powerful tool for DIHM systems operating under LPB conditions, providing accurate phase reconstructions with minimized noise. This advancement opens new possibilities for high-speed imaging of dynamic biological samples and imaging of samples with extremely low optical thickness. Future work could explore the further optimization and automatization of parameters for specific imaging scenarios and the application of IGA algorithm in other computational imaging systems. In particular, the IGA based algorithms should be possible to implement in other QPI systems, where the iterative phase retrieval algorithms from multiplexed data are employed (e.g., ptychography^63^ or Fourier ptychographic microscopy^64^).