Readout of fluorescence functional signals through highly scattering tissue

TL;DR

This paper demonstrates that functional fluorescence signals from sources beneath highly scattering tissue can be effectively retrieved using matrix factorization algorithms, overcoming previous limitations in non-invasive optical recordings.

Contribution

The study introduces a novel approach employing matrix factorization to extract functional signals through highly scattering tissue, enabling non-invasive optical brain activity monitoring.

Findings

Successful retrieval of fluorescence signals through scattering tissue

Effective demixing of signals from speckle patterns

Potential for non-invasive brain activity imaging

Abstract

Fluorescence is a powerful mean to probe information processing in the mammalian brain. However, neuronal tissues are highly heterogeneous and thus opaque to light. A wide set of non-invasive or invasive techniques for scattered light rejection, optical sectioning or localized excitation, have been developed, but non-invasive optical recording of activity through highly scattering layer beyond the ballistic regime is to date impossible. Here, we show that functional signals from fluorescent time-varying sources located below an highly scattering tissue can be retrieved efficiently, by exploiting matrix factorization algorithms to demix this information from low contrast fluorescence speckle patterns.

Figures

a). The result after background removal is shown in fig.[2](#S0.F2).a and supp.fig.1.c. Because of the relatively wide emission spectrum and the large size of the beads, the contrast of the pattern in fig.[2](#S0.F2).a is lower than unity, which would correspond to a fully developed speckle from a point-like monochromatic sourceGoodman ([2007](#bib.bib27)). In fact, we expect a speckle pattern with a contrast $C=1/\sqrt{N}$, where $N$ represent the number of independent degrees of freedom of the light after going through the scattering mediumGoodman ([2007](#bib.bib27)). These degrees of freed

Equations

$$\underset{\zeta ,\tau }{\min }\left|x(t)-\sum _{i=1}^r{\zeta }_i{\tau }_i(t)\right|=\underset{Z,T}{\min }\left|X-ZT\right|$$

Full text

In the last decades novel light-enabled tools established new paradigms in neuroscienceBoyden et al. (2005); Chen et al. (2013); Helmchen and Konnerth (2011), and among them the emergence of fluorescence functional indicators revolutionized the way to monitor information processing through the brain of different animal models, with unprecedented combination of contrast, resolution and specificityLivet et al. (2007); Weisenburger and Vaziri (2018). With this approach, optical resolution is often not paramount, and in general only a coarse (cell) resolution is neededPrevedel et al. (2016). Furthermore, when the location of neurons is known, it is possible to avoid slow raster-scanning techniques and image only the needed location at high frame-rate Iyer et al. (2006); Katona et al. (2012); Grewe et al. (2010); Prevedel et al. (2016); Bovetti et al. (2017).

However, brain tissues are usually opaque, and light emitted or delivered at depth in the brain is often quickly subject to multiple scattering events. This results in a loss of directionality after few scattering lengths, corresponding to a few hundred microns, and ultimately means that all wide field or scanning microscopy techniques fails at depth. While the brains of simple organisms are sufficiently small and/or transparent so they can be imaged in totality, for instance C.Elegans, drosophila or zebrafishWeisenburger and Vaziri (2018), mammalian brain, starting with its most common animal model, the mouse, is too large and too scattering to image in full. When imaging is performed in superficial layers, it is possible to implement wide field recording with multi-site multiphoton excitation Nikolenko (2008); Bovetti et al. (2017), or with wide-field excitation and a-posteriori demixing, exploiting the few forward scattered or ballistic photons as a seed to separate the individual neuron contributions Zhou et al. (2018); Nöbauer et al. (2017); Pégard et al. (2016). However, observing neuronal activity beyond a millimeter in the cortex or through the skull, is to date extremely challenging. In this depth range, in the multiple scattering regime, several techniques have been introduced to focus light and image using wavefront shapingHorstmeyer et al. (2015); Rotter and Gigan (2017). Fluorescence is conventionally considered very incoherent, so these techniques based on coherence do not straightforwardly apply. However, it has been shown that it is still possible to reconstruct a fluorescent object hidden behind a scattering medium, by analyzing spatial correlation within a single low contrast fluorescent speckle Katz et al. (2014); Hofer et al. (2018); Chang and Wetzstein (2018); Stern and Katz (2019); Xu et al. (2018). These techniques require thin media (with the so-called memory effect Judkewitz et al. (2015)) and limited object sizeHofer et al. (2018); Chang and Wetzstein (2018); Katz et al. (2014), restricting their application at depth in tissues.

However, precise neuron localization is not always needed to retrieve important insights on the brain function. This is the case for instance in some electrophysiological recording techniques, where activity of multiple neurons is often captured simultaneously, with only a coarse localization and no cellular sub-population selectivityHong and Lieber (2019). The use of genetically encoded fluorescence markers and optical readout would allow to expand such recording to precise targeting, minimal invasiveness, and single neuron resolutionLin and Schnitzer (2016); Livet et al. (2007); Helmchen and Konnerth (2011); Weisenburger and Vaziri (2018).

Here, we therefore choose to completely relax this imaging constraint: our goal is not to retrieve an image of a fluorescent object, nor localize it, but instead to retrieve the temporal activity from deeply buried fluctuating sources (ideally the functional activity of a set of neurons), by recording in wide-field their fluorescence. We rely on the fact that each source, after scattering through a thick opaque medium, will generate an extended but well defined spatial pattern at the detector, i.e. a speckle, which will be its unique footprint. This pattern will be modulated in time and summed incoherently with the other source’s footprints on a camera. Here, we show experimentally that it is possible to exploit these low contrasted fluctuating speckle patterns from extended fluorescence sources to extract functional signals even through highly scattering tissues, using an advanced signal processing algorithm, based on non-negative low-rank matrix factorization. We demonstrate the technique through a mouse skull, using large fluorescent sources and realistic temporal signals.

In order to test this approach, we designed an experimental setup providing a reliable ground truth of the light emitted from the fluorescent object, and the activity which they encode. We simulate the temporal activity of a small network of synthetic neurons made by $10\mu m$ fluorescent spheres (close to the size of common neurons bodiesBovetti et al. (2017)), where the emission spectra has been chosen to be close to common green fluorescent activity indicators, and using publicly available neuronal activity recordings and fluorescence indicator’s physiological modelsChen et al. (2013); Deneux et al. (2016). As described in methods, the temporal activity is generated by exciting the beads using a blue laser and a spatial light modulator (fig.1). The bead’s fluorescence emission then experiences the scattering through an ex-vivo mouse skull (thickness $\simeq 300\mu m$, and $l_{s}\simeq 40\mu m$ Soleimanzad et al. (2017)). The time-fluctuating low contrasted speckles are then recorded on a sCMOS camera for the analysis through a $10\times$ water objective and a standard fluorescence emission filter.

When a single bead is illuminated, the intensity pattern recorded at the camera shows a clear speckle structure (supp.fig.1a). The result after background removal is shown in fig.2.a and supp.fig.1.c. Because of the relatively wide emission spectrum and the large size of the beads, the contrast of the pattern in fig.2.a is lower than unity, which would correspond to a fully developed speckle from a point-like monochromatic sourceGoodman (2007). In fact, we expect a speckle pattern with a contrast $C=1/\sqrt{N}$, where $N$ represent the number of independent degrees of freedom of the light after going through the scattering mediumGoodman (2007). These degrees of freedom would result from unpolarized and spatially incoherent emission, since it originates from an extended object, and from independent spectral speckle components. When considering our real measurements $x(t)$, we can try to retrieve the temporal traces $\tau_{i}(t)$ and the spatial footprints $\zeta_{i}$ by finding the solution to the minimization:

[TABLE]

In this form the problem can be formulated as a low-rank factorization, where a matrix $X\in\mathbb{R}^{p\times t}_{>0}$ is approximated with two lower rank matrices $Z\in\mathbb{R}^{p\times r}_{>0}$ (the footprints) and $T\in\mathbb{R}^{r\times t}_{>0}$ (the time traces), where $r$ is the desired rank, $p$ are the pixels, and $t$ the frames. In our case, $r$ corresponds to the number of fluorescent sources which compose the recorded signal (which can be independently inferred from the measurements, see supp.fig.4). Since the extracted signals and their demixed speckle footprints are real-valued, the matrix factorization can take advantage of a non-negativity constrains with no loss of generalization. This minimization problem falls in the class of the Non-negative Matrix Factorization (NMF) framework, which has been already implemented with outstanding results in other functional imaging signal extraction techniques Pnevmatikakis et al. (2016); Nöbauer et al. (2017); Zhou et al. (2018), albeit for shallow depth when performed in wide field.

The random nature of the speckle generation, the strong independence of the $\zeta_{i}$ vectors, and the physiologically sparse temporal activity, allow the NMF algorithm to converge to a solution with an initialization based on Singular Value Decomposition (SVD) (supp.fig.5). The retrieved solution shows a remarkable correlation both with the known ground truth of the temporal activities and spatial footprint, which suggest that the retrieved solution is close to the global optimum (fig.2.c-d).

Since this approach relies only in the randomization of the wave-front performed by the scattering tissue, the relative axial position of the emitter should not compromise the ability to demix sources located at different depths. This property is demonstrated in the results displayed in fig.3, where are shown few representative traces of two set of 10 sources which have been artificially placed at different depth, $100\mu m$ apart (see methods).

In both fig.2.c and fig.3.b we can see the good correlation of the extracted traces with the ground-truths, which is quantified in fig.2.d-e. Another important point is the simplicity of the algorithm, which contrarily to implementation at low depth Nöbauer et al. (2017); Pnevmatikakis et al. (2016) does not require an initial guess of the source location (see methods). In practice, we believe the randomization itself (performed by the multiple scattering) actually helps to perform efficiently the factorization, as it is well known in signal processingMahoney (2011). Here, we report on successfully demixing 20 synthetic neurons (limited by the density of fluorescent beads in our sample). However, we believe, based on the SVD decomposition and residual image contrast, that the technique could probably scale to many more neurons, in particular exploiting longer acquisition sequences or adding priors on the temporal structure of the signals. Another important future direction will be to study whether one could use the footprints to extract information about the shape or localization of the sources.

In conclusion, we have shown that it is possible to retrieve the individual temporal traces of buried fluorescent sources, despite –and actually taking advantage of– the random scattering process performed by the tissue itself. Using realistic parameters for neuroscience samples, we demonstrate that it is possible to exploit low contrasted speckles, resulting from broadband and spatially extended fluorescence emission, to extract functional signals with a NMF-based algorithm, without the need of an initial guess, nor of any assumption on the temporal signal structure or spatial localization of the sources. Importantly, this technique does not rely on ballistic light, or in the presence of speckle correlation, thus is intrinsically adapted to highly scattering regimes. There are obviously many challenges to overcome to apply this technique in in-vivo situations (as background fluorescence, high number of sources, and sample movement), still, our work opens a unique avenue towards wide field functional imaging in highly scattering mediums at unprecedented depths.

Acknowledgments

The authors thank Laurent Bourdieu for providing the biological samples and for numerous discussions, Alipasha Vaziri and Tobias Nöbauer for useful suggestions, Saroch Leedumrongwatthanakun, Jonathan Dong and Antoine Boniface for constructive comments.

Additional information

This work was funded by H2020 European Research Council (ERC) (SMARTIES–724473). S.G. is a member of the Institut Universitaire de France.

Methods

Setup

Excitation light is produced by a 473nm DPSS laser (LSR-0473-PFM-00100-01, Laserglow Technologies, CA), expanded through a to fit a Digital Micromirror Device (DMD; DLP LightCrafter 6500, Texas Instruments, US-TX). Light is then focused into the sample with a $200mm$ lens (LA1708-A, Thorlabs) and a 20x (nominal) objective (Plan-NEOFLUAR 20x 0.5NA, Zeiss, DE; bottom OB in fig.1). A control optical path is used to collect ballistic light with a dichroic mirror (FF496-SDi01, Semrock, US-NY), a filter (MF525-39, Thorlabs), a $100mm$ tube lens (AC254-100-A, Thorlabs), and a CMOS camera (ACE2014-55um, Basler, DE). The sample is composed by $10\mu m$ beads (FluoSpheres F13081, Thermofisher scientific, US-MA), between two #1.5 coverslips. Beads emission spectra has been filtered to match common green emitting reporters, as GCaMP. Mouse skull is obtained from young adult mice. Light is collected with a 10x (nominal) objective (UMPlanFl 10x 0.3NA, Olympus, FR; top OB in fig.1), and is filtered either a $43nm$ FWHM barrier filter (MF530-43, Thorlabs; BF in fig.1) in most of the experiments, or a $10nm$ FWHM filter (FL05532-10, Thorlabs) if explicitly stated. A $200mm$ tube lens (AC254-200-A, Thorlabs) is then used to create the image at the sCMOS sensor of the main camera (EDGE 2.4, PCO, DE). An epi-illumination pathway has been set as well to support alignment procedures (supplementary figure 6).

Activity Traces

Synthetic fluorescence traces has been generated using spike activity from real available datasets acquired in mouse visual cortexChen et al. (2013), and calcium traces have been generated from these spikes using a GCaMP6s physiological model Deneux et al. (2016), and resampled at 10Hz. Pulse-width modulation was used to encode light intensity levels with the binary DMD amplitude modulation. A calibration routine has been written in Matlab (MathWorks, US-MA) to precisely match DMD pixels to the sample plane, and so the fluorescent beads positions. Acquisition, DMD control, and their synchronization is performed using Matlab (see script repositories for detailed implementation).

Data Acquisition

Imaging objective (upper OB in fig.1) has been focused $2mm$ from the skull surface, to obtain a proper sampling of the pattern. Every recording, the pattern produced by the distinct beads has been recorded one-by-one (fig.2.b), which provides the speckle footprint ground truth, and allows studying the correlation with the extracted footprints and the cross-correlations among them to estimate the memory effect (supp.fig.2). Exposure time has been set to 400ms. Frames has been cropped to the central zone of the background envelop. For figure3, we recorded both planes sequentially, then a-posteriori overlaid the recording, to emulate incoherent superposition of all neurons footprints. All the cross-correlation between traces and footprints has been evaluated as zero normalized cross-correlation to manage the transmission disomogenheities across the sample.

Algorithm

A proper spatial filter has been shown to be of crucial point to obtain a reliable demixing in the later analysis steps. A gaussian filter in the Fourier domain is used to remove structures with frequencies higher than the speckles grain size, assumed to be detection noise, and to remove the background envelope. The dataset has been previously sub-sampled in space reaching a shape of  100x100 pixels, then a singular value decomposition of the dataset has been performed. The resulting eigenvalue distribution has been clusterized using a k-mean algorithm, and the number of sources has been identified (supplementary figure 4). The factorization has been performed with decomposition/NMF class from the Scikit Learn Python libray. We didn’t use any sparsity constraints nor priors to retrieve the matrices. Initialization has been performed using a NNDSVD algorithmBoutsidis and Gallopoulos (2008). Factorization algorithm has been iterated 100 times, but faster convergence was achieved for most of the datasets (supplementary figure 5).

Data Availability

Full datasets are avaiable upon request.

Analysis scripts are available at: github.com/m0ro/SpeckledNeurons_analysis

Hardware control scripts are available at: github.com/m0ro/SpeckledNeurons_control

**Supplementary informations:

Readout of fluorescence functional signals through highly scattering tissue

**