The Next Dimension: Digital Holography for 3D Interferometric Scattering
Jaime Ortega Arroyo, Matz Liebel
TL;DR
This paper explains how digital holography can be used to improve the accuracy of measuring and tracking tiny particles in 3D.
Contribution
The paper introduces a detailed experimental framework for using off-axis digital holography in high-sensitivity nanosizing applications.
Findings
Off-axis digital holography offers advantages over traditional methods for interferometric scattering.
Digital postprocessing enables 3D particle tracking and aberration correction.
The approach is broadly applicable to nanosizing and particle characterization.
Abstract
We provide detailed experimental guidelines for implementing digital holography in the context of high-sensitivity interferometric scattering (iSCAT)-based nanosizing applications. Our approach relies on interferometry via the highly versatile off-axis implementation of digital holography, which offers key advantages over more traditional strategies. After a brief theoretical discussion of off-axis holography and its differences and similarities with iSCAT, typical experimental implementations and digital data-processing steps are presented. Key experimental parameters and strategies to achieve optimal performance are also highlighted. Following these experimental aspects, we focus on digital postprocessing routines that enable digital refocusing and 3D particle tracking as well as pupil function aberration correction. We then conclude with a few examples highlighting the broad…
Genes, proteins, chemicals, diseases, species, mutations and cell lines named across the full text — each resolved to its canonical identifier and authoritative record.
Click any figure to enlarge with its caption.
1
2
3
4
5
6
7
8- —HORIZON EUROPE European Innovation Council10.13039/100018703
- —HORIZON EUROPE European Research Council10.13039/100019180
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Holography and Microscopy · Advanced X-ray Imaging Techniques · Advanced Electron Microscopy Techniques and Applications
Introduction
All-optical label-free sizing and sensing approaches are highly relevant for addressing both fundamental and applied challenges. Applied technologies, such as nanoparticle tracking analysis or mass photometry, are widely used in analytical laboratories for routine nanocharacterization. ?,? Fundamentally, single-particle methods are prime candidates for answering biophysically relevant questions, especially when ensemble averaging masks the underlying dynamics. ?−? ? ? Historically, observations were often based on so-called darkfield observations where only light scattered by nano-objects of interest is detected. ?−? ? ? ? However, it was soon realized that interferometric approaches offer key advantages as they boost small scattering signals and exhibit favorable particle size-dependent signal scaling. ?−? ? ?
A very successful implementation, especially in the biophysics community, is interferometric scattering (iSCAT)? microscopy. iSCAT is a form of inline holography where the reference wave is generated as a back-reflection off an interface, typically the glass/air or glass/solvent interface of the coverglass holding the sample. iSCAT achieves high sensitivity, down to the single protein level, ?,? and allows high-speed observations.? As “mass photometry”, these aspects are combined to enable landing assays that label-free infer the mass of single proteins in a highly linear and sensitive fashion. iSCAT’s inline nature makes it readily compatible with fluorescent imaging modalities? but also comes with specific drawbacks related to nontrivial signal scaling? and twin-image problems.? Additionally, backscattering-based approaches struggle when transitioning from the Rayleigh- to the Mie-scattering regime, and parasitic back reflections generated inside the microscope objective render a conceptually easy-to-implement methodology experimentally very challenging to adopt, especially for larger fields-of-view or when targeting absolute sensitivity limits.
The aforementioned drawbacks can be eliminated by moving away from iSCAT’s inline configuration. So-called off-axis holography, or interferometric scattering, also relies on interference between two electric fields but at an angle, that is, in an off-axis configuration.? The two fields are typically generated externally, which allows choosing appropriate experimental parameters that eliminate twin images, nontrivial signal scaling, and parasitic back reflections.? While being a popular methodology in the broader optics community, highly sensitive, iSCAT-type, nanoscale measurements are rarely reported. In this tutorial review, we will discuss how the concepts of off-axis holography and iSCAT can be seamlessly combined in a highly synergistic fashion to yield quantitative, ambiguity-free, nanoscale observations over extended 3D volumes at sensitivities comparable to iSCAT microscopes.
Theoretical Considerations
Interferometric techniques such as iSCAT, inline, or off-axis holography rely on the interference between two electric fields. Conceptually, they are all identical. As such, we will refer to the fields involved as signal, E s, and reference, E r, fields, irrespective of the specific technique. The former contains the image information of interest, and the latter serves as a reference and is often assumed to carry no additional information. When spatially and temporally overlapping at a detector, these fields interfere, thus generating a so-called hologram, I holo
With E (E*) being the complex (complex conjugate) of the electric field. For the sake of simplicity, we have omitted the physical constants in all equations relating the hologram intensity at the detector and the electric fields. Using E = A e^–iφ^, with A being the electric field amplitude and φ its phase, we can rewrite Equation as
where Δφ = (φ_s_ – φ_r_) is the phase difference between the signal and reference field. Equation broadly describes all two-field interference experiments. As such, it describes approaches relying on inline holography, such as interference reflection or iSCAT microscopy. ?−? ? What makes off-axis holography distinct is its ability to computationally isolate the interference terms, A s A _r _e^–iΔφ^ or A_s_A r e^ iΔφ^, from the intensity terms, A s ^2^ + A r ^2^, and separate the amplitude and phase information. ?,? In other words, off-axis holography isolates the signal’s complex electric field and unlocks computational postprocessing routines that are difficult to combine with inline detection schemes due to the twin-image problem.?
Results and Discussion
Off-Axis
Holography: General Implementation
As the name suggests, off-axis holography relies on interference between noncollinearly traveling signal and reference fields. Figurea shows a possible implementation. A beamsplitter generates illumination, E illu, and reference, E r, fields, typically from a spatially coherent light source such as a laser. E illu interacts with the sample of interest, and a microscope objective collects the signal field, E s, containing illumination light alongside sample-scattering, E sca. A lens then propagates E s onto a camera, placed conjugate with the sample plane, where interference with E r occurs.
How to perform off-axis holography. (a) Minimum-complexity experimental implementation of off-axis holography. (b) Wavefront schematic explaining off-axis induced oscillatory modulation using plane waves. (c) How to extract phase and amplitude images from a hologram; simulated data. The absolute value of the complex k-space is shown.
The angle between E s and E r allows separating the interference and amplitude square terms, ?,?,? as outlined in Figureb using plane waves. In brief, the angle between the waves results in a position-dependent linear phase gradient. Assuming an angle in the y-dimension, we can rewrite Equation as
With Δφ = Δφ_sample_ + ay, where Δφ_sample_ is the sample-induced phase difference between E s and E r and ay is a linear, y-dependent phase gradient. Equation shows that the interference term is spatially modulated with a. In other words, a Fourier transformation allows isolating it in momentum-, or k-space. Figurec summarizes the off-axis workflow, from acquired hologram to isolated phase and amplitude images. In a first step, the as-acquired hologram is Fourier transformed into k-space. Here, the amplitude square terms and the interference terms are separated due to the linear phase gradient. The amplitude square terms are located around 0,0 in k-space, and two interference terms are visible, a direct result of the complex and complex conjugate (Equation), which inverts the phase and with it the off-axis induced phase gradient. Hard-aperture selecting one interference term followed by shifting its center to 0,0 and inverse Fourier transforming yields the complex interference term in image space, which can be separated into its amplitude and phase components.
Off-Axis Holography: Magnification and Interference
Angle
To successfully implement off-axis holography following the workflow outlined above, it is important to ensure that the interference terms do not overlap with each other or the amplitude square terms. This condition can be satisfied by adequately choosing an image magnification as well as the angle between the k-vectors of E s and E r. Without going into details, we recommend employing a magnification that ensures that the nominal detector pixel size, Δpx, corresponds to
with NA being the numerical aperture. Although not being the most space-bandwidth efficient implementation, this configuration allows separating all terms along the k-space diagonal, thus making it relatively straightforward to implement. For a detailed discussion, we refer the interested reader to Dardikman et al., ?,? who provide an excellent summary on the topic alongside strategies to improve the space-bandwidth product.
While it is possible to calculate the necessary interference angles,? we generally determine the correct interference angle experimentally by systematically adjusting it while observing a Fourier transformation of the hologram. Care should be taken not to choose too large angles, a possibility given that aliasing can make a too-large angle indistinguishable from the correct configuration. To avoid this scenario, we initially keep both E s and E r in the same horizontal plane and only adjust the vertical plane. Following this first step, we then carefully adjust the horizontal dimension by walking E r up and down via two adjustable mirror mounts while monitoring the k-space locations of the interference term. This approach allows detecting and hence avoiding aliasing. If a diagonally placed interference term (see k-space in Figurec) does not exhibit equal horizontal and vertical displacement, then the larger displacement needs to be corrected.
Off-Axis Holography: Crucial
Experimental Details
The blueprint presented in Figurea, in principle, allows straightforward implementation of off-axis holography. However, achieving high-quality measurements requires carefully balancing a few crucial experimental parameters, as discussed in detail in Figurea. These aspects are related to coherence and wavefront properties that can be nonobvious but need to be accounted for when designing an experiment. From our experience, interferometric stability is generally of no concern in off-axis holography as long as beam heights, integration times, and path lengths are kept within reasonable, microscopy-suitable limits.? The most important stability requirement is that air currents and acoustic vibrations are eliminated, e.g., the optical table should be covered, and sources of convection, such as power supplies, etc., should not be mounted in the vicinity of the optical paths. Gradual, often nanometric, path length changes between individual image acquisitions only result in relative phase shifts, which are measured and hence removable by simply subtracting a constant. As such, we do not discuss these aspects.
Experimental details and temporal coherence. (a) Experimental implementation and key points of interest where nonobvious experimental aspects can complicate off-axis holography. (b) Interference contrast as a function of optical path length difference between signal and reference arm measured for a diode-pumped solid state laser (532 nm, blue line), a single-mode laser diode (520 nm, brown line), and a femtosecond laser (515 nm, yellow line). Zero delay is at the center of the traces. (c) Left: Simulated and measured interference contrasts were obtained for “standard” off-axis geometry using lasers with varying temporal coherence lengths. The diagonal line marks the direction of the maximum angle between the signal and reference. Right: comparison of interference-amplitude contrasts for “standard” and grating-based off-axis holography along the diagonal line indicated on the left. (d) Impact of wavefront curvature mismatch on the k-space representation of the interferogram.
Off-Axis Holography: Optical Path Length Matching
Even for most CW lasers, the optical path length difference between the signal and the reference arm must be carefully matched. Figureb highlights this aspect by comparing path length-difference dependent interferograms recorded for three typical light sources of decreasing temporal coherence length: a diode-pumped solid state laser (DPSS), a single-mode laser diode, and a frequency-doubled femtosecond laser (Methods and Experimental). Around zero delay difference, all sources show satisfactory interference, followed by a rapid loss of interference. Both the DPSS as well as the diode show surprisingly short coherence lengths, with the diode being almost comparable to the femtosecond source. The DPSS shows a slow beating pattern, whereas the laser diode shows recurring interference maxima at >1 mm delay intervals. What summarizes these observations is that path length control is crucial. Mismatches on the <1 mm scale can result in dramatic signal loss, a fact that complicates the experimental setup but comes with an important benefit: coherence gating, which conveniently eliminates parasitic interferences with back reflections from or scattering off optical components. Experimentally, we advise systematically controlling the path length difference via a manual translation stage and selecting the position of maximum interference contrast. This aspect is especially crucial for diode lasers, where the distant local interference maxima can show dramatic contrast reduction when compared to the true “time-zero”. Generally, the position of maximum interference contrast corresponds to the correctly matched configuration.
Off-Axis Holography: Adjusting
Wavefront Tilt
At short coherence lengths, matching the wavefront tilt at the camera chip becomes important, especially for large detectors where signal-to-reference angle-induced distance differences easily exceed >100 μm. Figurec compares the expected and measured interference term amplitudes. The latter is directly obtained via off-axis holography following the workflow outlined in Figure. The “expected” amplitudes are computed based on individually measured A s ^2^ and A r ^2^ images as the product of the square roots of the two measurements. As can be seen, the short-coherence light sources exhibit reduced amplitudes along the direction of interference, a direct result of the angle-induced path length difference. To circumvent this problem, it is possible to generate the reference as the first diffraction order off a grating that is relay imaged onto the camera plane. ?,?
Figurec highlights how a grating-based approach allows eliminating loss of interference (Methods and Experimental), a strategy that we successfully used for ultrabroadband fields covering the entire visible spectral range with <2 μm temporal coherence length.?
Off-Axis Holography: Wavefront Curvature
Matching
Finally, the wavefront curvature of the signal and reference should be matched at the detector plane. ?,? For infinity corrected objectives, plane wave reference fields are often a good starting point. For finite-conjugates, focusing the reference at a distance from the detector corresponding to the tube length is a good approximation. Experimentally, we typically transmit the laser through the off-axis setup and then vary the position of a collimation/focusing lens of the reference field until the size of the interference term in k-space is the smallest. We next mount a somewhat concentrated nanoparticle sample onto the microscope to generate a high signal-to-noise ratio projection of the back-focal-plane (BFP) onto the interference term in k-space and then insert a darkfield stop into the objective’s BFP. For correctly matched wavefront curvature, the interference term should look like the BFP (Figured). Depending on the level of residual mismatch, the BFP is either sharp or defocused. Fine adjustment of the reference curvature, by moving the collimation lens, based on the BFP appearance, allows straightforward system optimization. Noninfinity corrected objectives often require a diverging reference wave. Given the low cost of such optics, we advise to simply mirror the microscope in the reference arm to generate a correctly matched reference wave.
Off-Axis Holography: Experimental Flavors
Thus far, we have focused our discussion on image-space holography (Figurea) as an intuitive extension of darkfield or iSCAT microscopy. From a workflow perspective, one optimizes the microscope following established routines and then adds holographic capabilities, which makes the implementation somewhat straightforward. Computationally, this modality extracts phase and amplitude information by relying on the position-momentum Fourier relationship. It is therefore also feasible to conduct momentum, or k-space, off-axis holography, which ultimately yields image-space images.? Figureb schematically describes the implementation. Rather than placing the camera into a conjugate image plane, it is placed into a conjugate Fourier plane, e.g., at the position of the BFP. Interference is analogous to image-space holography, but now a single Fourier transformation is sufficient to isolate the complex image-space interference terms. Experimentally, both approaches have advantages and disadvantages that need to be carefully balanced when selecting the best implementation.
Experimental flavors. (a) Image-space holography. (b) k-space holography. (c) Forward vs backscattering geometries and particle size-dependent signal scaling. (d) Illumination at large k-vectors eliminates parasitic back-reflection and mitigates optic damage, but choosing the correct polarization is crucial. The simulations (c, d) were performed using MiePlot v4.6.21, , assuming a surrounding refractive index of 1.33 and particle refractive index of 1.5 at a wavelength of 532 nm. The polarization-dependent angular scattering amplitude simulations were performed using the same parameters and a particle radius of 10 nm.
Off-Axis Nanoscopy: Illumination Geometries and Signal Levels
Off-axis holography allows freely selecting the illumination geometry, which warrants a careful evaluation of angle-dependent scattering amplitudes. Figurec,d discusses a few illumination geometries that we commonly use in our laboratories. Transmission and reflection, e.g., 0 and 180° angle of incidence, are widely used, corresponding to brightfield and interference reflection microscopy. The former, e.g., forward detection, accurately recovers particle sizes when measured based on scattering amplitudes with , whereas the latter rapidly underestimates the size: a direct result of the transition from Rayleigh to Mie scattering (Figurec).? This effect not only leads to size ambiguities but also has advantages. For instance, it reduces scattering signals of possibly present larger contaminations. Alternative illumination geometries, not applicable to inline detection, are highlighted in Figured. These implementations do not propagate the illumination light through the microscope objective, which eliminates all parasitic reflections and allows for dramatically increasing the illumination intensity. Especially the latter feature is highly desirable for large field-of-view observations. Beyond particle size-dependent Mie scattering, as discussed previously, the illumination polarization has to be carefully adjusted when employing oblique- or side-illumination schemes. Figured highlights the dramatic scattering amplitude differences between parallel and perpendicular polarized illumination. Similarly, potential polarization rotations need to be accounted for to ensure that signal- and reference-waves interfere at the detector. Finally, when employing light sources of short temporal coherence lengths, the position-dependent path length differences for the latter geometries might result in unfavorable signal scaling and need to, hence, be carefully characterized.
Computational Postprocessing: z-Propagation
Following the detailed experimental description, we will now focus on the second main pillar of off-axis holography, which is a crucial advantage over alternative schemes: computational image postprocessing. The capabilities of computational postprocessing are unlocked once the complex electric field has been isolated (Figurec). For nanosizing applications using particle suspensions, z-propagation, or digital refocusing, allows reconstructing 3D volumes from a single 2D acquisition (Figurea) ?−? ? as long as the sample is sufficiently sparse. The excellent review by Memmolo et al. provides a general introduction to the topic.? We implement 3D propagation via the so-called angular spectrum method,? which is ideally suited for dealing with the large scattering angles encountered and conveniently implemented via computationally fast Fourier transformations. In brief, the image-space field is transformed into k-space and then multiplied by the following propagation kernel
where k m = 2πn/λ, with n being the refractive index of the propagation medium, λ the wavelength, and z the propagation distance. The discretized spatial frequencies are (k _ x _, k _ y _) = 2π(x,y)/(MΔx) for (−M)/2 ≤ x,y ≤ M/2, with Δx being the magnified pixel size of the imaging system after Fourier-extracting the interference term.
Numerical propagating. (a) Holographically recorded 2D image planes can be propagated to different z-positions by using a so-called propagation kernel. (b) An experimentally acquired hologram (dark plane) is propagated to reveal a 3D volume of the sparse sample composed of 80 nm diameter Au nanoparticles immobilized in a 4% agarose gel. (c) 3D localizing the nanoparticles while systematically varying the agarose sample to objective distance by means of a z-piezo shows that precise 3D localization over extended distances is feasible. Bottom inset: an in-focus and 40 μm out-of-focus nanoparticle show identical piezo-steps. (d) Comparison of particle localizations for particles present near the image plane and far out-of-focus. Essentially identical localization precisions are obtained, thus validating holographic 3D tracking applications. All experiments were performed via backscattering-based off-axis holography in a darkfield configuration.
This propagation kernel can be understood as a lens function with a wavenumber cutoff determined by the refractive index of the propagation medium, which allows reconstructing large 3D volumes. An example is shown in Figureb, where we computed a 3D image stack from a single plane recording of an 80 nm diameter Au nanoparticle doped agarose gel using a numerical aperture 1.2 water immersion objective (Methods and Experimental). To evaluate how the z-position of a given particle with respect to the image plane impacts the localization precision, we varied the sample to objective distance by means of a closed-loop piezo, taking three 42 nm steps followed by a larger step (Methods and Experimental). For each step, we 3D localized all particles based on volumetric representations, as the one shown in Figureb (Methods and Experimental). A qualitative comparison between the individual particles’ locations reveals that the nanometric steps are detectable for both physically in-focus particles as well as 40 μm out-of-focus particles (Figurec). To further quantify the z-dependent localization precision, we computed the mean position change, using all particle localizations, and compared it to the changes detected on a particle-by-particle level. Figured shows that no significant localization precision difference is visible for particles located around the physical image plane as compared to far out-of-focus candidates.
When propagating over large volumes, it is important to keep in mind that signal loss might occur for far out-of-focus particles, which ultimately degrades sensing performance and impacts the recovered scattering amplitudes. The reasons are 2-fold. First, a large defocus means very strong wavefront curvature. As a result, particle scattering might reach the detector, but its interference might be incorrectly detected due to aliasing effects. Second, the scattered light radially spreads, which means that some might be lost as it no longer reaches the detector, a common scenario given the limited detector size. Unsurprisingly, this effect is especially severe for objects near the image edge. Combined, both effects might ultimately result in xyz-position-dependent amplitude scaling, especially for large defocus. Experimentally, a suitable propagation range can be estimated by visually inspecting the resulting xy images at a given z-position. When reaching the cutoff range, effects such as point-spread-function blurring or a sudden drop in observed particle densities, as compared to in-focus images, are a clear indication. Importantly, as long as the particles can be localized, the experimentally known parameters, such as image size, imaging optics, and z-propagation, in principle allow renormalizing all scattering amplitudes based on the extracted xyz-positions using a full physical model of the image formation and propagation process.
Computational Postprocessing: Aberration Correction
Defocus is an optical aberration, and it should thus come as no surprise that other forms of aberrations, such as coma or astigmatism, can be computationally corrected for. Compared with the z-propagation discussed above, the challenge is to determine the optical aberrations prior to removing them. Our strategy relies on isolating individual point scatterers to infer pupil aberrations. An intuitive example using immobilized nanoparticles and a noncoverglass-corrected microscope objective is presented in Figure. In brief, when the imaging system is used as intended, we observed near aberration-free images as shown in Figurea. Fourier transforming an image containing only one particle reveals the residual, minimal pupil aberrations. Upon inserting a slab of glass between the sample and the objective, followed by manual refocusing, we note a considerably degraded point-spread function alongside marked spherical pupil aberrations (Methods and Experimental). We remove these aberrations via a two-step approach based on Zernike polynomials. First, we coarsely estimated the aberrations and subtracted them in k-space with the goal being to eliminating the visible phase-wrapping toward high k-vectors, which complicates direct fitting approaches. For this initial estimate, we rely on fits using one-dimensional cuts through the center of the BFP, which can be disambiguity-free unwrapped as long as the darkfield-stop region of zero information is ignored. Following subtraction, we obtain wrapping-free pupil aberrations, which are then fitted using the first 21 Zernike polynomials, resulting in an essentially aberration-free pupil plane (Figureb). Back Fourier transformation into image-space indeed confirms near-perfect aberration correction, in line with both experimentally obtained and theoretically expected point-spread-function cross sections (Figurec).
Aberration correction. (a) Experimentally obtained nonaberrated and intentionally aberrated images of a single Au nanoparticle on glass. The pupil plane of the particle imaged “as specified” indicates a near aberration-free imaging system, whereas the same system with glass added between the sample and the microscope objective shows dramatic pupil aberrations. The missing areas in k-space are due to a darkfield stop. (b) Aberration removal based on coarsely estimating the pupil aberrations, to eliminate phase-wrapping, followed by linearly fitting the sum of 21 Zernike polynomials. (c) Comparing aberrated, aberration-corrected, and theoretically expected point-spread functions demonstrates near-perfect computational aberration correction. All experiments were performed via backscattering-based off-axis holography in a darkfield configuration.
To experimentally implement the aberration correction outlined above on nonideal, or volumetric, images, we typically use hard-aperture-isolated multiple individual nanoparticles in image space. We then set their phase at the center of the point-spread-function to zero, followed by shifting all particles to the same location, ideally DC. We then average all particles and inverse Fourier transform, followed by the steps outlined in Figure. Once the correction pupil phase, φ_correct_, is obtained, the original hologram is aberration corrected by multiplying its k-space representation by e^(−iφ_correct_)^.
Applications
Combined, the steps outlined above allow establishing a working off-axis holographic microscope (Figure) and optimizing key performance parameters related to the optical design and light source (Figure). Figure explains how to balance illumination parameters and sample choices, followed by a summary of key computational postprocessing concepts dedicated to z-propagation (Figure) and aberration correction (Figure). To showcase how these concepts translate to real-world scenarios, we conclude with three dedicated applications discussing nanosizing, particle-motion-based thermal gradient mapping, and the study of photoinduced phenomena.
Applications: Nanosizing
Off-axis holography is ideally suited for size and composition characterization of synthetic or natural nanomaterials, such as metallic or dielectric nanoparticles, tailored nanometric drug-delivery vectors, or extracellular vesicles. At the limits of sensitivity, surface-based inline holography in a backscattering configuration is arguably the method of choice. It enables extended observation times, which allow achieving sufficient signal-to-noise ratios to detect tiny nano-objects such as single proteins. ?,?,?,? However, these approaches suffer from important drawbacks. First, the difficulty of separating amplitude square and interference terms can lead to signal ambiguities for particles exhibiting scattering amplitudes comparable to the reference-field amplitude (eq).? Second, when transitioning from the Rayleigh to the Mie scattering regime, scattering signals no longer scale with the particle size (Figure). Third, as scattering amplitudes are a function of both particle size and composition, it is difficult to distinguish contaminations from analytes of interest. Finally, precise focus control is often necessary, which is costly. Off-axis approaches overcome these drawbacks and do not require costly focus control, making them ideal candidates for commercially viable turn-key instruments for the analysis of unknown or heterogeneous nanoparticle suspensions.
To quantitatively measure heterogeneous clinical nanoformulations, we devised k-space holography (Figuresb and ?a–c),? which relies on interference in the BFP rather than in real space. Compared to real-space imaging, this modality projects the scattering signal of all nanoparticles onto all camera pixels, thus dramatically boosting the achievable dynamic range by approximately six orders of magnitude. A Fourier transformation is sufficient to recover real-space images from the k-space holograms (Figurea). Using this approach, we simultaneously measured Au nanoparticles covering a diameter range of 20–250 nm, corresponding to a > 10^5^-fold change in scattering intensity (Figureb). The technology enabled directly quantifying SkOV3-derived extracellular vesicle distributions (Figurec) based on a low-cost setup, surface-capture, and external signal-calibrations using silica nanoparticles.
Nanosizing using off-axis holography. (a) k-space interferogram (left) alongside its Fourier transformation (right) of a sample containing 20 nm diameter Au nanoparticles as observed with a numerical aperture 0.7 microscope objective. (b) Scattering signals alongside Gaussian fits (dashed lines) for Au nanoparticles with diameters in the 20–250 nm range. (c) Size distributions of extracellular vesicle samples with a noise cutoff around 25 nm. (d) Volumetric 3D particle tracking for advanced nanocharacterization enabled by off-axis holography. (e) Holographic nanoparticle tracking analysis (holoNTA) distinguishes nanoparticles based on scattering amplitudes and hydrodynamic diameters. (f) holoNTA is well-suited for characterizing heterogeneous mixtures. Panels a–c are reprinted or adapted with permission from ref . Copyright 2020 American Chemical Society. Panels (d–f) are reprinted or adapted with permission under a Creative Commons Attribution 4.0 International License from ref . Copyright 2023 American Chemical Society. Panels (a–c) were performed using an oblique illumination geometry without passing the objective, Panels d–f in forward-scattering darkfield.
While powerful, scattering signal-based approaches relying on calibrations are unable to identify the nature of the underlying particles. In other words, a particle of unknown refractive index cannot be correctly sized. This aspect is especially important in the context of extracellular vesicles, where it is difficult to distinguish larger protein aggregates from vesicles. To address these limitations, we took advantage of the holographically extended volume of observation, which allows robust 3D single-particle tracking over long observation times (Figured). Holographic nanoparticle tracking analysis (holoNTA), the combination of scattering-based characterization with holographically extended 3D nanoparticle tracking analysis (NTA), yields two parameters. ?,? As such, it provides robust particle characterization: 3D tracking yields hydrodynamic diameters which, combined with scattering signals, allow inferring particle sizes and material composition. Figuree highlights the strength of holoNTA when applied to particles exhibiting comparable scattering signals but dramatically differing compositions. More specifically, holoNTA was able to distinguish Au and SiO_2_ nanoparticles of varying sizes based on the two-parameter observations, and also robustly analyzed heterogeneous samples (Figuref). Compared to near-surface techniques, ?,? holoNTA’s extended volumes eliminate the need for high-speed acquisition, which greatly reduces associated equipment cost and, more importantly, eliminates the need for precise nanoparticle localization.?
Applications: Thermal Gradient Mapping
One of the main advantages of off-axis holography versus its inline counterparts is straightforward single-shot access to quantitative phase information from the sample. Beyond the advantages of propagation and aberration correction already described above, off-axis holography is ideally suited to leverage this property to extract information about the surrounding microenvironment. This is possible because changes to the local microenvironment, in the form of either ionic species,? temperature,? or buffer composition,? give rise to differences in the local refractive index, which in turn can be measured experimentally as relative phase changes. In other words, an off-axis holography microscope operates as a high-resolution (diffraction-limited) wavefront sensor, which, for instance, can be used to reconstruct the 3D thermal gradient landscape.
Figurea illustrates the working principle of a holographic temperature gradient sensor, whereby an incident plane wave accumulates an overall phase difference as it travels across a localized refractive index gradient caused by a temperature gradient in the sample volume. Experimentally, we generated this thermal gradient by irradiating the sample with a pump beam resonant with the optical absorption of efficient light to heat transducers. In our case, we used plasmonic nanoparticles, which allowed us to fabricate an all-optical reconfigurable nano-to-microscale heat source. In this specific example, we used a transmission-based off-axis system (Figurea,c) combined with a pump–probe scheme to demonstrate how illumination of a single subdiffraction-limited plasmonic structure (<250 nm) leads to a detectable phase difference. Exploiting established analytical solutions relating temperature fields with measured optical path length differences,? we retrieved the underlying 3D temperature gradient map.
*Thermal gradient mapping using off-axis holography. (a) Working principle of a wavefront-based temperature gradient sensor using a pump–probe approach. (b) Representative orthogonal projections of the motion of 1 μm tracer beads in the presence of a thermal gradient contained within a microchamber of nominal height of 50 μm oriented perpendicular to the direction of gravity as shown in the schematic diagram. (c) Data analysis workflow to extract drift velocity vectors that capture both particle and fluid dynamics from the 3D tracks of the tracer particles. First, each tracer particle track is segmented into pairwise instantaneous velocity vectors, u. Next, the distribution of all instantaneous velocity vectors within a given voxel is computed. Then, from the distribution of u, the ensemble average flow velocity vector, U
avg , for each voxel is extracted, thereby suppressing Brownian motion contributions. Finally, this process is repeated for all voxels within the imaged volume. (d) Correlative 3D temperature and drift velocity field maps capturing both particle and fluid dynamics upon inducing a thermal gradient inside the microchamber depicted in (b). Panels (c, d) are reprinted or adapted with permission under a Creative Commons Attribution 4.0 International License from ref . Copyright 2021 Springer Nature. All experiments were performed via forward-scattering-based off-axis holography in brightfield configuration.*
Notably, because these wavefront-based measurements are intrinsically in situ, one can then apply holography-enabled 3D single-particle tracking to study how local perturbations to the microenvironment (e.g., presence of a temperature gradient) affect both single nanoparticle and fluid dynamics. Using tracer beads (1 μm), we captured the dynamics of thermally driven phenomena, such as thermophoresis, convection, and thermoosmosis (Figureb). More specifically, the single-particle tracking velocimetry approach outlined in Figurec allowed decomposition of individual particle trajectories into instantaneous 3D displacement vectors (u). Ensemble averaging localized vectors over individual voxels allowed extracting high-resolution drift velocity (* U * _ avg _) maps alongside the induced temperature gradient (Figured). Using this approach, we identified experimental parameters, such as the microchamber height, size, and number of heat sources, and orientation of the microchamber with respect to gravity, that could tune the contribution of each of the thermally driven phenomena to the observed dynamics. This nano- to microscale insight on thermally driven phenomena allows informed engineering of a variety of microfluidic functionalities, such as long-range transport? or reconfigurable thermal barriers that emulate physical ones.?
While simple and sensitive, the range of applications for wavefront-based temperature gradient sensors remains limited by temperature retrieval algorithms. These algorithms are derived from models that assume systems in steady-state with heat sources located in the same plane and a temperature field smoothly decaying inversely proportional to the distance from the heat sources. These assumptions, together with the need for an imaging model relating phase to the temperature fields, can be entirely circumvented by adding a k-vector scanned illumination to the off-axis holographic system, thus converting it into an optical diffraction tomography (ODT) one. The main hallmark of ODT versus off-axis holography is the direct retrieval of the 3D complex refractive index map over the entire imaged volume.? Combining pump–probe ODT measurements with a look-up table that relates the refractive index to the temperature of a specific material unlocks time-resolving nonsteady state (transient) temperature maps without the need for any models. ?−? ? Alike off-axis holography, recent advances in ODT have enabled high-speed volumetric tracking of single particles.?
Applications:
Photoinduced Changes
Off-axis holography allows single-shot multiplexing through the use of multiple illumination and reference waves. These capabilities have been broadly exploited to enable, for example, color, polarization, or temporal multiplexing. ?−? ? ? Recently, we realized that this concept can be extended to retrieve high-speed signal modulations from long-duration camera exposures, thus enabling holographic lock-in wide-field imaging, ?,? a modality that had previously been restricted to point-detection. These advances allow visualizing rapidly occurring photoinduced processes at low signal levels on conventional cameras.
Figurea summarizes the working principle of a holographic lock-in camera, where a pump–probe experiment is combined with a real-space off-axis holographic microscope employing two reference waves. The probe beam continuously illuminates the sample, while both the pump as well as both reference waves are rapidly modulated. Synchronizing pump and reference-wave modulation allows spatially encoding pump_ON_ and pump_OFF_ signals into the same images. Fourier filtering, analogous to single-reference holography, recovers the distinct images from the multiplexed hologram (Figureb). ?,? When combined with ultrashort pulses, so-called phototransient holography allows studying photoinduced changes on femto- to nanosecond time scales. A distinct advantage over the previously discussed modalities is that phototransient imaging infers additional chemical information through resonant excitations. To demonstrate these capabilities, we conducted initial proof-of-concept experiments on 60 nm Au and 100 nm latex nanoparticles, both of which exhibit comparable scattering amplitudes (Figurec). When illuminated with a 400 nm pump, only the Au nanoparticles showed phototransient signal changes, a direct result of hot electron generation via the surface plasmon resonance of Au. The elevated electron temperatures result in distinct spectral shifts of the resonance, which are readily detectable in the visible spectral range.? By changing the pump–probe time delay, it was further possible to follow the nanoparticles’ thermalization dynamics on femto- to picosecond time scales (Figurec).? Finally, combining the 3D tracking capabilities with phototransient microscopy allows one to study photoinduced dynamics in freely moving objects. Figured shows a 3D trajectory of a freely diffusing 100 nm diameter Au nanoparticle alongside its transient dynamics following photoexcitation.
Monitoring photoinduced dynamics by off-axis holography. (a) Experimental schematic and working principle of all-optical phototransient wide-field imaging with a lock-in camera. (b) Signal-retrieval scheme for phototransient imaging. (c) Phototransient imaging allows distinguishing resonant (Au) and off-resonant (latex) materials based on their photoinduced differential scattering signals. (d) Phototransient imaging and pump–probe delay-dependent spectroscopy of a freely diffusing 100 nm nanoparticle. Panels a–d are reprinted or adapted with permission under a Creative Commons Attribution-NonCommercial 3.0 Unported License from ref . Copyright 2022 The Royal Society of Chemistry. All experiments were performed via forward-scattering based off-axis holography in darkfield configuration.
Conclusions
Summary and Future Trends
Looking ahead, we envision several exciting avenues for future development. The speed and throughput of GPU-based computation have increased dramatically over the past years, developments that have dramatically benefited holographic imaging processing. Their FFT-based nature means that the ever-improving advanced parallelization schemes immediately expand holographic capabilities both in terms of speed and also in terms of algorithm-complexities. This development is likely to soon allow real-time 3D hologram analysis and visualization using desktop-compatible GPUs. These capabilities, in turn, will enable sophisticated data processing and background removal approaches, thus further enhancing off-axis holography’s sensing and sizing capabilities. Enabled by these and further technical innovations, we believe that real-time free-flow analysis of even single proteins should be within experimental reach.
By trading ultimate sensitivity for larger volumes of view, via low numerical aperture lenses, it will become possible to observe individual nano-objects for minutes without relying on surface binding. These capabilities will enable real-time studies of nanoscale reactions and photochemistry under relevant experimental conditions, an exciting toolbox that is expected to considerably contribute to the growing insight obtained through so-called operando studies.
Phototransient holography offers exciting opportunities for wide-field studying of nanoscale photoinduced dynamics in real time. Beyond the currently employed plasmonic systems, tunable excitation sources will allow applying this promising technology to detecting, analyzing, and studying dielectric and even biological matter with chemical specificity.? We expect contributions in the broader context of wide-field photothermal approaches, employing NIR or MIR excitation sources. Here, nanosecond implementations have already uncovered exciting biological phenomena that are difficult to assess with alternative means. ?,? Expanding such observations to the temporal limits of intramolecular vibrational energy redistribution? and nanoscale heat-diffusion ?,? is likely to contribute valuable insight and will, potentially, allow developing novel diagnostically relevant imaging modalities.
Taken together, we believe that the fusion of holographic imaging modalities with ultrasensitive nanoscopy will facilitate fundamental studies and enable the development of commercially viable and relevant platforms. This highly promising combination of digital imaging approaches with traditional optics allows replacing costly hardware with exponentially improving in silico solutions, and expands the palette of available hardware and software tools to address the future challenges in characterizing heterogeneous nanoparticle samples.
Methods
and Experimental
Temporal Coherence Length Experiments
We used three distinct light sources: A 532 nm DPSS (CW532–100 Roithner Lasertechnik GmbH), a 520 nm laser diode (PD-01298 Lasertack GmbH) and 515 nm light derived by frequency doubling the 1030 nm femtosecond output of an amplified Ytterbium laser system (200 fs, PH2–20W-SP Pharos Light Conversion) using a 1.5 mm β-barium borate (BBO) crystal cut at 23.4 deg, Type I.
The interferograms reporting on temporal coherence lengths were measured via a conventional Michelson interferometer using a stepper motor to systematically change the path length difference (M230.25, Physik Instrumente (PI) SE& Co.KG).
The grating-based off-axis interference experiments derived their reference wave as a relay-imaged first diffraction order off an 80 lines/mm Ronchi grating (Edmund Optics), oriented along the diagonal of the camera chip (a2A2840–48umBAS, Basler AG).
The impact of wavefront curvature was assessed by systematically changing the collimation properties of a reference wave in a 520 nm laser diode (PD-01298 Lasertack GmbH) based darkfield off-axis holographic microscope.
Numerical Propagation Experiments
We prepared 80 nm diameter Au nanoparticle suspensions (BBI Solutions) in 4% low-melting-point agarose and cast hydrogels on a 1.5H coverglass. A backscattering-based off-axis holographic microscope based on a 532 nm DPSS (CW532–100 Roithner Lasertechnik GmbH) equipped with a numerical aperture 1.2 objective (UPLSAPO60XW/1.20, Olympus) acquired z-stacks by systematically changing the focal-plane position with a closed-loop z-piezo (MIPOS 100 SG, Piezosystem Jena).
Holograms were extracted via established Fourier filtering methods followed by numerical propagation via the angular spectrum method, using 400 nm steps over a total distance of 80 μm (±40 μm). For 3D localization, we segmented the resulting 3D intensity maps into regions of interest based on local maxima. The particle-containing segments were then propagated onto a finer z-spacing of 100 nm, covering ± 2 μm. We then determined the particles’ xy-center-of-mass and fitted a parabola along the z-dimension to infer the z-position.
Aberration Correction Experiments
A 520 nm laser diode (LDM-520–100-C, Lasertack)-based off-axis holographic microscope equipped with a long working distance microscope objective (Mitutoyo MY100X-806) operated in forward-scattering with a darkfield mask positioned between the objective and the sample recorded the images. 60 nm diameter Au nanoparticles (BBI Solutions) were deposited onto a 1.5H coverglass and mounted with the nanoparticles facing the objective. Nonaberrated images were recorded in said configuration. The aberrated images were recorded by inserting a 1 mm thick fused silica window between the sample and the objective. To access the pupil aberrations, we isolated the interference term via established Fourier filtering methods, hard-aperture filtered a single nanoparticle in image space, shifted it to (0, 0), and extracted the pupil aberrations, free of linear phase ramps, in k-space.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Filipe V.Hawe A.Jiskoot W.Critical Evaluation of Nanoparticle Tracking Analysis (NTA) by Nano Sight for the Measurement of Nanoparticles and Protein Aggregates Pharm. Res.20102779681010.1007/s 11095-010-0073-220204471 PMC 2852530 · doi ↗ · pubmed ↗
- 2Young G.Hundt N.Cole D.Quantitative mass imaging of single biological macromolecules Science 201836042342710.1126/science.aar 583929700264 PMC 6103225 · doi ↗ · pubmed ↗
- 3Yildiz A.Forkey J. N.Mc Kinney S. A.Myosin V Walks Hand-Over-Hand: Single Fluorophore Imaging with 1.5-nm Localization Science 20033002061206510.1126/science.108439812791999 · doi ↗ · pubmed ↗
- 4Schmidt T.Schütz G. J.Baumgartner W.Gruber H. J.Schindler H.Imaging of single molecule diffusion Proc. Natl. Acad. Sci. U.S.A.1996932926292910.1073/pnas.93.7.29268610144 PMC 39736 · doi ↗ · pubmed ↗
- 5Manzo C.Garcia-Parajo M. F.A review of progress in single particle tracking: from methods to biophysical insights Rep. Prog. Phys.20157812460110.1088/0034-4885/78/12/12460126511974 · doi ↗ · pubmed ↗
- 6Moerner W. E.A Dozen Years of Single-Molecule Spectroscopy in Physics, Chemistry, and Biophysics J. Phys. Chem. B 200210691092710.1021/jp 012992 g · doi ↗
- 7Kusumi A.Nakada C.Ritchie K.Paradigm Shift of the Plasma Membrane Concept from the Two-Dimensional Continuum Fluid to the Partitioned Fluid: High-Speed Single-Molecule Tracking of Membrane Molecules Annu. Rev. Biophys. Biomol. Struct.20053435137810.1146/annurev.biophys.34.040204.14463715869394 · doi ↗ · pubmed ↗
- 8Ando J.Nakamura A.Visootsat A.Single-Nanoparticle Tracking with Angstrom Localization Precision and Microsecond Time Resolution Biophys. J.20181152413242710.1016/j.bpj.2018.11.01630527446 PMC 6302141 · doi ↗ · pubmed ↗