A silicon photonic modulator neuron

TL;DR

This paper reports the development of a silicon photonic modulator neuron capable of network interactions, demonstrating key neuron behaviors, and enabling fully integrated photonic neural networks on existing silicon platforms.

Contribution

It introduces the first silicon photonic neuron with behaviors like transfer function configurability, fan-in, inhibition, and cascadability, advancing neuromorphic photonics.

Findings

Demonstrated transfer function configurability

Achieved fan-in and inhibition behaviors

Proved autaptic cascadability of the neuron

Abstract

There has been a recently renewed interest in neuromorphic photonics, a field promising to access pivotal and unexplored regimes of machine intelligence. Progress has been made on isolated neurons and analog interconnects; nevertheless, this renewal has yet to produce a demonstration of a silicon photonic neuron capable of interacting with other like neurons. We report a modulator-class photonic neuron fabricated in a conventional silicon photonic process line. We demonstrate behaviors of transfer function configurability, fan-in, inhibition, time-resolved processing, and, crucially, autaptic cascadability -- a sufficient set of behaviors for a device to act as a neuron participating in a network of like neurons. The silicon photonic modulator neuron constitutes the final piece needed to make photonic neural networks fully integrated on currently available silicon photonic platforms.

Full text

††preprint: APS/123-QED

I Introduction

Renewed interest in neuromorphic photonics has been heralded by advances in photonic integration technology Orcutt et al. (2012); Lim et al. (2014); Thomson et al. (2016), roadblocks in conventional computing performance Marr et al. (2013); Hasler and Marr (2013), the return of neuromorphic electronics Merolla et al. (2014); Furber et al. (2014); Benjamin et al. (2014); Meier (2015); Davies et al. (2018), and the inundation of machine learning (ML) with neural models Schmidhuber (2015). Neural networks have held some role in ML (e.g. image and voice recognition, language translation, pattern detection, and others) since the 1950s Von Neumann (1956); Rosenblatt (1958). They fell out of favor in the 90’s because they are difficult to train.

Over the past decade, neural network models have decisively retaken the helm of ML under the alias of “deep networks” Najafabadi et al. (2015). There are three main reasons: 1) major algorithmic innovations Lecun et al. (1998); LeCun et al. (2015), 2) the Internet: an inexhaustible source of millions of training examples, and 3) new hardware, specifically graphical processing units (GPUs) Pallipuram et al. (2012). Central processing units (CPUs) are woefully inefficient at evaluating these models because they are centralized and instruction-based, whereas networks are distributed and capable of adaptation without a programmer. GPUs are more parallel, but, today, even they have been pushed to their limits Diamond et al. (2016).

Today’s demand for evaluating neural network models necessitates new hardware. High-tech juggernauts and research agencies have heavily invested in massively parallel application-specific integrated circuits (ASICs) for evaluating neural network models more efficiently, notably IBM Merolla et al. (2014), HP Pickett et al. (2013), Intel Davies et al. (2018), Google Jouppi et al. (2017); Graves et al. (2016), the Human Brain Project Markram (2012), and DARPA SyNAPSE Cassidy et al. (2013). Some of these architectures aim to be ML number crunchers Miyashita et al. (2017); Jouppi et al. (2017), and others have enabled novel neuroscientific tools Pfeil et al. (2013); Friedmann et al. (2013) and previously unforeseen low-power mobile applications Tsai et al. (2016).

The primary performance driver for the neuromorphic electronics community is computational power efficiency; speed is a secondary consideration. Neuromorphic electronics have largely focused on biological-timescale neural networks: kHz (with one 10MHz exception Pfeil et al. (2013)). They universally rely on digital time- and event-multiplexing Mundy (2016), which means they cannot simply run faster by turning up the clock. Nevertheless, there are compelling applications for neural networks with nanosecond latency. Some applications could be offline (i.e. number crunching) such as accelerators for deep network training and inference; others could be online (i.e. real-time) such as pattern detectors for wideband radio frequency (RF) signals and feedback controllers for systems subject to short-timeconstant instabilities. Moving beyond the nanosecond will require moving beyond purely electronic physics.

Photonic physics exhibit properties distinct from those of electronics in terms of multiplexing, energy dissipation, and cross-talk. These properties are favorable for dense, high-bandwidth interconnects Rakheja and Kumar (2012) in addition to configurable analog signal processing Weiner (2011); Pérez et al. (2017); Liu et al. (2016). Consequently, neuromorphic photonic systems could operate 6–8 orders-of-magnitude faster than neuromorphic electronics Shastri et al. (2018) with potentially higher energy efficiencies Ferreira de Lima et al. (2017). Optical neural interconnects based on field evolution in free-space Brunner and Fischer (2015); Psaltis and Quio (1990), holograms Goodman et al. (1978); Asthana et al. (1993), and fiber Hill et al. (2002) have been shown but were not widely adopted, in part because they cannot be integrated on a chip and thereby scaled robustly and manufactured cheaply. Analog interconnects integrated on a silicon photonic platform have been shown Carolan et al. (2015); Shen et al. (2017), but these interconnects are optically coherent, and a cascadable photonic neuron that regenerates phase from layer to layer has not been proposed.

* * *

In this work, we fabricate and demonstrate a silicon photonic modulator neuron. It consists of a balanced photodetector directly connected to a microring (MRR) modulator. We demonstrate that this device possesses the necessary capabilities of a network-compatible neuron: fan-in, high-gain optical-to-optical nonlinearity, and indefinite cascadability – properties never before demonstrated together in a single integrated device. Furthermore, we demonstrate optional, but useful, capabilities of transfer function configurability, inhibitory fan-in, pulse compression, and time-resolved processing. To establish empirical evidence of indefinite cascadability, we employ the methodology of Tait et al. (2017): inducing an observable bifurcation in an autapse circuit.

Network-compatible photonic neurons are optical-in, optical-out devices that must be able to 1) convert multiple independently weighted inputs into a single output (i.e. fan-in), 2) apply a nonlinear transfer function to the weighted sum of the inputs, and 3) produce an output capable of driving multiple other neurons, including itself (i.e. cascadability). Research in neuromorphic photonics has seen an abundance of semiconductor lasers that exhibit nonlinear transfer functions, particularly those of a spiking neuron Nahmias et al. (2013); Coomans et al. (2011); Van Vaerenbergh et al. (2013); Brunstein et al. (2012); Selmi et al. (2014); Romeira et al. (2016); Nahmias et al. (2016); Deng et al. (2017); Peng et al. (2018), reviewed in Prucnal et al. (2016). Nevertheless, all of the conditions of network-compatibility have yet to have been conclusively demonstrated in a single device (with exception of the fiber laser in Shastri et al. (2016)), and much of this research overlooks fan-in and/or cascadability entirely. There are fundamental reasons why fan-in, nonlinearity, and cascadability are difficult to achieve in all-optoelectronic devices, discussed further in Sec. IV.1. The modulator neuron addresses these issues using an optical/electrical/optical (O/E/O) signal pathway with a lumped, intermediate electronic connection.

The neuron shown here is integrable with the silicon photonic neural network demonstrated in Tait et al. (2017). Modulator neurons and the MRR weight network combine to form a neuromorphic photonic architecture called broadcast-and-weight Prucnal and Shastri (2017); Tait et al. (2014), diagrammed in Fig. 1. In this architecture, photonic neurons output optical signals with unique wavelengths. These are wavelength-division multiplexed (WDM) and broadcast to all others, weighted, and detected. Each connection between a pair of neurons is configured independently by one MRR weight Tait et al. (2016a, b), and the WDM carriers do not mutually interfere when detected by a single photodetector. In the context of prior work, the MRR modulator neuron illustrates a path to fully integrated photonic neural networks compatible with currently available silicon technology. This compatibility is a crucial element of scalability, cost, and feasibility.

Silicon photonics presents an opportunity for optical technologies to venture beyond pure communication links and specialized co-processing boxes. Recent advances have brought about a stable academic/industrial ecosystem surrounding silicon photonics Soref and Bennett (1987); Soref (2010), propelling technology road-mapping Thomson et al. (2016), standardized multi-project wafer runs Lim et al. (2014); Orcutt et al. (2012), broadened accessibility to academic research Chrostowski and Hochberg (2015), and economies of scale previously enjoyed solely by microelectronics. Despite its origins in cutting the cost of mid-reach communications, silicon photonic integration could be instrumental to large-scale photonic information processing concepts – beyond what could be considered in fiber, III-V, or holographic platforms.

II Methods

II.1 Device Description

Theory of operation

The modulator neuron is an optical-to-electrical-to-optical (O/E/O) device consisting of two balanced photodetectors (PDs) connected electrically to a microring (MRR) modulator. It that takes two balanced optical inputs, subtracts them electronically, and remodulates a signal onto a new wavelength (Fig. 2(a)). The output signal is a nonlinear function of inputs determined by the electro-optic transfer function of the modulator.

Fig. 2(a) shows the neuron circuit diagram. The WG labeled “pump” directs a continuous-wave (CW) laser signal at $\lambda_{n}$=1544.8nm to the MRR modulator neuron. When modulated, this signal serves as the neuron’s optical output. The PDs are reverse-biased and convert impinging light at all wavelengths into photocurrents. Photocurrent from the positive PD adds to the injected current, while photocurrent on the negative PD shunts away some of the injected current coming from the bias port. These photocurrents are combined with a bias current $I_{b}$ such that their sum affects the refractive index of the MRR modulator via free-carrier injection. This index change affects its transmission at the pump wavelength, and ultimately modulates the amount of light transmitted at the output port of the neuron. The MRR resonance wavelength can also be thermally tuned by a in-situ heater with a current $I_{h}$, used for coarse wavelength locking and fine bias control.

Fabrication

Fig. 2(b) shows a false-color optical micrograph of the fabricated neuron. Waveguides (WGs), shown in yellow, are formed in the 220nm thick silicon layer. They are patterned to be 500nm wide except in long, straight segments, WGs are widened to 3$\mu$m to minimize sidewall scattering. The MRR modulator consists of a circular WG with a designed radius of 11.5$\mu$m. It is coupled to one WG with a gap of 200nm. The WGs containing active devices are partially etched to a 90nm thick pedestal (light green), which hosts the dopants.

Doping profiles of active regions are diagrammed in Fig. 2(c). The MRR has two doped regions: an N*+/N/N+* junction to act as an ohmic heater and an N*+/N/P/P+* junction to act as a high-speed modulator. Phosphorous and boron concentrations are the same as in Baehr-Jones et al. (2012). A germanium layer is deposited and patterned (dark green) along with additional dopants that create a vertical P-I-N photodiode (Fig. 2(d)). Silicon oxide is then deposited, vias are etched, and aluminum traces are deposited (red) – this sequence is repeated to form a second wiring layer (orange). Metal traces are patterned to break out the device electrical ports to a row of probe pads.

II.2 Device characterization

The optical behavior of the MRR modulator and its response to different effects is characterized by the profile of its resonance dip. We examine the optical transmission spectrum of the MRR modulator in response to three independent quantities:

Heater current bias ($I_{h}$) 2. 2.

Modulator current bias ($I_{b}$) 3. 3.

Optical power into the IN+ and IN– ports ($P_{+}$ and $P_{-}$, respectively)

The spectra are plotted in Fig. 3, taken by a transmission spectrum analyzer (Apex AP2440A). The bare resonance is seen in Fig. 3(a) as the blue curve. The modulator’s PN junction is forward-biased by currents $I_{b}$ such that dynamic inputs induce an index change through free carrier injection. When electrical bias is applied, the peak blue shifts to the left, seen over different rows. When heat is applied, it red shifts to the right, seen over the different columns. Orange, green, and red curves show different optical input states, described below.

Comparing the blue curves of (a-d) with (e-h), we see that thermal tuning does not significantly affect the response besides shifting the absolute wavelength with efficiency of 0.24nm/mW. This is desirable so that the thermal degree of freedom can be used as an independent parameter to lock the MRR onto a WDM channel of interest without affecting the electro-optic response. In contrast, the electrical bias parameter does effect the depth and quality factor (from 14.5k to 3.5k) of the peak. As a result, the electrical bias can be used to configure the response, while the combination of electrical and thermal parameters can be used to maintain the desired wavelength of interest. The electrical tuning efficiency below diode threshold is zero because no free carriers are injected. Above diode threshold, the tuning efficiency is at most 0.26nm/mA.

Two lasers are used to characterize the O/E/O response. When optical inputs are non-zero, the effect is the same as modifying the electrical bias because they deflect the current seen on the modulator. Orange curves represent optical power (10.dBm at facet) on the IN+ port with the IN– laser turned off. The photocurrent adds to the bias current, causing a blue-shift. The fiber-to-fiber insertion loss through the chip is 18.dB, so we estimate that the input GC represents 9.0dB of loss. Therefore, the optical power reaching PD+ would be 1.26mW. This causes a deflection equivalent to 0.96mA, resulting in an estimated PD responsivity of 0.76A/W, slightly less than in other reports Fard et al. (2016). The green curve is the opposite case: IN+ laser off and IN– laser at 10.dBm at facet. The red curve is with both lasers on. The fact that red curves match up with blue (lasers off) curves is significant because it shows that the +/– optical inputs cancel out. This means that excitatory and inhibitory effects can be realized by switching the port of impinging inputs. Microring weight banks Tait et al. (2016a, 2018a) explored in other work were specifically designed to perform this task of directing multiple wavelength channels between the ports of a balanced photodetector.

The optical-to-optical gain is measured by comparing the amplitude of the modulation swing on the $\lambda_{1}$ input to the amplitude of the modulation swing on the $\lambda_{n}$ output coming from the neuron. It is found to be 2.16$\times 10^{-2}$, dominated by fiber-to-chip insertion loss measured at 18.0dB. Removing the coupling loss results in an on-chip gain of 1.36. Gain scales in proportion to the pump power, which was held artificially low during this measurement to eliminate the chance of the pump influencing the carrier concentration or temperature of the modulator. Pump power was measured as –5.5dBm at the chip facet, an estimated –14.5dBm arriving at the neuron, which is less than one hundred times weaker than it could be under realistic assumptions.

We use injection modulation because we found that the depletion effect was too weak to demonstrate the desired results with the non-optimized modulators we fabricated. Injection modulation with a forward-biased junction is slower than depletion modulation with a reverse-biased junction. Injection modulation bandwidth can reach up to 6.25GHz Xu et al. (2007) compared to depletion modulators shown up to 40GHz Novack et al. (2013). All of the signal processing concepts of modulator-class neurons shown here hold over different types of modulation mechanisms.

II.3 Experimental setup

The multi-channel input generator used in the below experiments is shown in Fig. 4. All time-varying signals coming in and out of the chip are optical. Two input wavelengths ($\lambda_{1}$=1546.4nm and $\lambda_{2}$=1548.2nm) are wavelength-division multiplexed and power modulated by a fiber Mach-Zehnder modulator (MZM). Distinct input signals are then created by a relative delay, $\Delta T$, between these wavelengths following the method of Tait et al. (2015). Laser pumps for the external inputs and the neuron itself come from a distributed feedback (DFB) laser array (ILX 7900B). The $\lambda_{2}$ signal can be switched between the excitatory and inhibitory ports.

There are three signal generators used in the below experiments, two analog (a.k.a. synths) and one binary. A simple, slow waveform generator (HP 8116A) is used to acquire transfer functions (Sec. III.1) and autapse behavior (Sec. III.5). The 8116A offers control of sawtooth waveforms that can be used to separate rising and falling aspects. Burst inputs are generated by a Rohde and Schwartz SMBV 100A VG (R&S), which is used in Secs. III.1, III.2, and III.3. The R&S effectively yields trains of return-to-zero (RZ) pulses of varying amplitude. Binary pulsed inputs used in Sec. III.4 are generated by a pulse pattern generator (PPG) (Anritsu MP1761B). The PPG provides the highest instantaneous bandwidth but the least control over waveforms.

The neuron’s output is coupled off-chip, detected, and observed in a sampling oscilloscope (Tektronix DSA8300). Between the output coupler and scope, there is a signal-to-noise enhancement stage, not diagrammed, consisting of an erbium doped fiber amplifier (EDFA), optical bandpass filter at $\lambda_{n}$, discrete photodetector (Discovery Semiconductors, Inc. DSC-R405ER), and, for low-bandwidth experiments, an electrical low-pass filter. The pre-chip subsystem contains two EDFAs and polarization controllers that are not shown. All of the above instrumentation is controlled remotely via lightlab, a free software python package Tait et al. (2018b).

The chip containing the neuron is placed on a temperature-controlled alignment stage and kept in place by a back-side vacuum. 4 fibers of a V-groove array are aligned to grating couplers leading to the IN+, IN–, PUMP, and OUT ports. DC bias signals are applied through 6 probe tips in an array (GGB MCW-26-8146). Power and ground signals are derived from a power supply, and tunable biases, $I_{b}$ and $I_{h}$, are derived from two Keithley 2400 current sources.

III Results

We conduct a series of experiments on this silicon photonic device to support the claim that it exhibits the three properties necessary to act as a network-compatible neuron (nonlinearity, fan-in, cascadability), in addition to several optional but useful properties. Sec. III.1 demonstrates nonlinear optical-to-optical conversion and transfer function configurability. Sec. III.2 demonstrates high-bandwidth operation and pulse compression. Sec. III.3 demonstrates fan-in and inhibitory fan-in. Sec. III.4 demonstrates time-resolved spike processing that is excitatory and inhibitory. Sec. III.5 demonstrates indefinite cascadability through the observation of bistability in an autapse circuit.

III.1 Transfer functions

The photodetector-modulator system exhibits a variety of nonlinear optical-to-optical transfer functions that are relevant for a wide variety neural processing tasks. To obtain these responses, the optical input is modulated by a slow rising sawtooth waveform at 200kHz, derived from the HP synth. The type of response depends on the wavelength offset between the pump and the MRR resonance. To change the response, the MRR resonance is shifted by varying the heater bias. Corresponding time-domain behavior is then shown using a 100MHz burst from the R&S generator.

Fig. 5 shows six response shapes, each relevant in different areas of neural processing and machine learning. They are obtained under different biasing conditions. This variety indicates that the MRR modulator neuron can yield nonlinear configurability, to complement the configurability of the network’s linear weights. The sigmoid shapes of Figs. 5(a, b) are commonly used in recurrent Hopfield networks for nonlinear optimization Hopfield and Tank (1985). They are obtained by biasing at the maximum slope point. The rectified linear unit (ReLU) shape of Fig. 5(c, d) is widely used in feedforward machine learning networks today, i.e. in multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs) Jouppi et al. (2017). Positive and negative ReLUs are obtained by biasing slightly off-resonance, either above or below the pump wavelength. A network that combines sigmoid and ReLU neurons is well-suited to solve nonlinear optimization problems with constraints, some of which are reviewed in Wen et al. (2009). The peaked transfer function of Figs. 5(e, f) are known as radial basis functions (RBFs). When biased on-resonance, the RBF is centered at zero, resulting in a quadratic or rectifying transfer function. The off-centered RBF is obtained by biasing off-resonance with high quality factor and a strong input amplitude. RBFs are commonly used for ML based on support vector machines Cortes and Vapnik (1995).

These nonlinear responses stem from the electro-optic transfer function of the modulator. The photodetectors’ O/E response is linear and the modulator’s E/O response is a strongly nonlinear, peaked Lorentzian. It is apparent that the different responses correspond to different pieces of the MRR modulator’s Lorentzian peak shape. To some extent, the type of response observed depends on the amplitude of the input, which is not ideal. The ideal ReLU continues increasing indefinitely, and the ideal sigmoid saturates indefinitely. We note however, that every neuron has some limitation to its range of inputs, whether it be imposed by register bit depth (digital) or state bleaching (analog, including biological).

III.2 Response to high-bandwidth inputs

The response of the neuron to a much faster input is shown in Fig. 6. The input RF envelope is a 25.0ns pulse of a 1.0GHz carrier, which is modulated on an optical carrier at $\lambda_{1}$. The outputs indicate that, under different biasing conditions, the neuron can either reproduce or apply nonlinear transformations to a fast input. The linear response of Fig. 6(b) is obtained by biasing on the maximum slope point and driving with a signal weaker than that needed to saturate the sigmoid of Fig. 5(b). This demonstration is significant, in part, because it is a faithful conversion of an RF signal from one wavelength, $\lambda_{1}$, to another, $\lambda_{n}$. A 20% voltage gain is shown, although this is not necessarily representative of the fully integrated case because gain is affected by fiber-to-chip insertion loss and fiber EDFA configuration.

In Fig. 6(c), the ReLU of Fig. 5(b) was biased below its elbow. Viewed on the burst level, it exhibits a thresholding phenomenon resulting in a compressed burst width of 20.4ns (19% compression). As seen in the zoomed in traces of Fig. 6(d-f), the ReLU also exhibits a pulse compression effect. The 1GHz input can be viewed as a train of 500ps full-width half maximum (FWHM) pulses that are turned into a train of 324ps pulses (36% compression).

Pulse compression is an important behavior for maintaining the integrity of pulses cascaded over multiple stages of neurons. It occurs because the leading and lagging pulse edges take on values that lie below the ReLU elbow. Both of these thresholding/compression behaviors are due to the positive third-order nonlinearity possessed by the ReLU. It is more common for optical devices to exhibit saturating behaviors with negative third-order nonlinearities, for example, those subject to two-photon absorption and laser diodes well above the lasing threshold. If saturating devices were used in a neural network, the pulse FWHM from stage-to-stage would increase (i.e. degrade) indefinitely.

III.3 Response to multiple inputs

We then consider the case of two inputs at different wavelengths, $\lambda_{1}$ and $\lambda_{2}$. We demonstrate fan-in behavior across wavelengths – the ability to perform a summation operation across multiple wavelengths while simultaneously converting them to a single wavelength, $\lambda_{n}$. Fan-in is central to the idea of network-based processing, so this feature is particularly important to demonstrate directly in order to claim a device as a “photonic neuron.”

The setup for addition and subtraction are shown in Fig. 4. The input envelopes are delayed versions of one another, i.e. $B(t)=A(t+\Delta T)$. For subtraction, the two inputs are sent into complementary ports of the neuron’s balanced PD. For addition, they are multiplexed and sent into the same port. This is represented by the excitatory/inhibitory switch in Fig. 4. The modulating signal, a 900MHz RF carrier that is amplitude modulated (AM) at 50MBaud, is chosen to demonstrate several behaviors of interest to RF signal processing, discussed below. The delay, $\Delta T$, is adjusted so that the RF carrier waves are in-phase and the modulating bit pattern of [1, 1, 0] is delayed by one bit period.

Figure 7(a,b) shows the inputs $A(t)$ and $B(t)$ after detection. Figure 7(c) is the optical output of the MRR neuron when these signals are multiplexed into the IN+ port. This illustrates basic optical fan-in of two WDM signals onto a single wavelength. Figure 7(d) is a more complex case showing excitatory fan-in followed by a nonlinear rectifying conversion from the electronic to optical domain. Figure 7(e, f) illustrates inhibitory behavior where the inputs counteract one another – more than just an inversion of the excitatory case. Figure 7(e) is with a linear neuron transfer function, and Fig. 7(f) is with a rectifying transfer.

The fact that input optical signals effect changes in the output optical signal is significant because that output could, in principle, be fed to other neurons; furthermore, the fact that multiple signals can be “weighted” by positive and negative values and their sum then influencing the output is an indicator that the MRR neuron can be networked with multiple inputs and outputs. Furthermore, the photodetector-modulator device can be viewed as a wavelength converter with higher gain than all-optical approaches Nozaki et al. (2018). Wavelength conversion on a conventional silicon platform has uses outside of neuromorphic photonics.

The signals themselves were chosen to show very simple signal processing tasks for typical AM radio signals. Linear fan-in is used in RF processing for dimensionality reduction and principal component analysis Tait et al. (2018c). Followed by a rectifying stage, this system effectively acts as a multi-channel envelope detector (i.e. $(A+B)^{2}$ and $(A-B)^{2}$). The expansion of these squares yields a $\pm 2AB$ term – a measure of correlation. In radio signal processing, some algorithms for channel estimation are based on calculating correlation to update estimation parameters Kutty and Sen (2016); Tait et al. (2018c). Very fast measurements of correlation could accelerate the convergence time and tracking rate of those algorithms.

III.4 Response to pulses

We examine the response of the neuron to two pulsed signals and show that the modulator neuron can be made to exhibit enhancing and saturating nonlinearity as well as inhibition in the pulsed domain. The significance of using pulses is the demonstration of time-resolved processing. When the magnitude of the response is dependent on the precise timing of inputs, the neuron is then capable of implementing tasks that involve temporally-coded signals, for example, temporal pattern detection and polychronization Izhikevich (2006).

In this experiment, the input is derived from the PPG producing a pulse doublet with $2.0\text{\,}\mathrm{ns}$ width and $30\text{\,}\mathrm{ns}$ (15-bit periods) inter-pulse delay. This signal is modulated onto $\lambda_{1}$ and $\lambda_{2}$, which are then delayed separately. The inter-channel delay is adjusted to correspond to the inter-pulse delay, such that the middle pulses coincide. At the same time, the leading and lagging pulses show the non-coincident response for comparison. The inputs post-delay are shown in Fig. 8(a, b). Fig. 8(c) compares the enhancing, saturating, and inhibitory cases to the linear sum of inputs (black trace). The traces are normalized to 0.5 at the first non-coincident pulse. This means that the linear sum at coincidence is 1.0.

Pulse coincidence detection is shown in the blue trace of Fig. 8(c). The coincident pulse peaks at 1.57, meaning that coincidence is over-emphasized. Enhancement is observed when the transfer function corresponds to the ReLU in Fig. 5(c), biased in the flat region. One pulse is not sufficient to reach the elbow, while two pulses are. We note that the single-pulse suppression is not complete because their bandwidth is near the 3dB bandwidth of the neuron. Coincidence detection is considered to be an essential property of nonlinear units within a pulsed neural network. A positive third-order nonlinearity resulting in pulse compression (shown in Fig. 6) is also essential for cascadability in a pulsed neural network because it counteracts pulse width spreading. In the enhancement experiment here, output pulses were not compressed. They have a FWHM of 3.8ns (90% spread) due to the limited bandwidth of injection modulation. Spreading can be decreased by using longer pulses or faster modulation, such as depletion-mode.

Pulse saturation is shown in the orange trace in Fig. 8(c). Again, the output is normalized to the leading pulse. In this case, the coincident pulses evoke about the same response as the non-coincident ones. The response corresponds to the inverse ReLU in Fig. 5(d), biased in the elbow region. The negative deflection around 20ns is a spurious artifact due to carrier relaxation.

Pulse inhibition is shown in the green trace in Fig. 8(c). Signal IN 2 is switched to the IN– port of the neuron. The non-coincident pulses can be seen to produce positive (leading pulse) and negative (trailing pulse) effects. When the pulses coincide, the output vanishes. This result indicates more than an inversion of the response from the last experiment, but the simultaneous capability for complementary excitable and inhibitory responses. In a real network, fibers cannot be switched. Instead, wavelength channels can be switched between IN+ and IN– ports by thermally configurable optical passives Tait et al. (2016a).

III.5 Cascadability

In order for neurons to be cascadable, signals must maintain their integrity from stage to stage indefinitely. The condition of gain cascadability can be expressed as the existence of an operating point where large-signal gain is one and small-signal, differential gain, $g$, is greater than one. The condition of physical cascadability is expressed, for photonic neurons, as the input and output both being optical and at the same wavelength. These conditions can be observed using a self-afferent neuron – also known as an autapse. The autapse is a nonlinear feedback system whose input and output are the same signal by definition. As derived in Appendix VI, such a system becomes bistable when its differential gain is greater than unity. Therefore, an observation of a transition between mono/bi-stability is equivalent to a demonstration of cascadability. The significance is that bifurcations are qualitative changes readily observable in experiment. Here, we construct an autapse circuit and observe therein this bifurcation, providing evidence that the MRR modulator neuron is indefinitely cascadable.

The autapse setup is shown in Fig. 9(b). There is one external input that is modulated by a kHz-timescale sawtooth waveform that is either rising or falling. The output of the neuron at $\lambda_{n}$ is fed back via fiber and multiplexed with this external signal. Then both are fed into the positive PD. Inline with the feedback pathway is a polarization controller (PC), manually variable feedback attenuator ($W_{F}$), optical tap to the oscilloscope, and EDFA to counter fiber-to-chip insertion loss.

Fig. 9(c) shows the baseline input-output relation for the case where $W_{F}=0$. The output (blue) is a regular function of the input, as seen in blue in Fig. 9(h). Figures 9(d-g) show the rising and falling cases where $W_{F}=1$. The output is different depending on the direction of the input (black). Figure 9(h) shows each output plotted vs. input, where colors correspond to the left panels. In Figs. 9(d-g), the biasing state is the same, whereas bias is different in Fig. 9(c) because the average input optical power is less. These two different qualitative behaviors reveals the presence of a cusp bifurcation between $W_{F}\in[0,1]$.

The experimental setup with a fiber in the feedback loop differs substantially from a fully integrated autapse shown in Fig. 1. These caveats are shared by Tait et al. (2017) and are discussed in Sec. IV.3.

IV Discussion

IV.1 Claiming cascadability and fan-in

Many all-optical and optoelectronic elements exhibit nonlinear or neuron-like input-output transfer functions, but this is not a sufficient condition for said device to be capable of networking with other like devices. Optics faces special challenges in satisfying the critical requirements of cascadability and fan-in, which we will refer to together as network-compatibility. These fundamental challenges were expressed by Keyes and Goodman more than 30 years ago Keyes (1985); Goodman (1985), and they remain challenges today. Above, we have argued that bifurcation in an autapse provides sound empirical evidence of both. Due to their historical difficulty and central importance, we believe that there should be a high burden of proof placed on proposed photonic neurons to clearly demonstrate cascadability and fan-in. Here, we will consider how other works have treated these concepts.

There are several physical properties that pose barriers to cascadability Keyes (1985). In optical devices whose nonlinear mechanism is based on semiconductor carrier-mediated cross-gain modulation (XGM) or Kerr effect, the controlled signal (probe) affects the material properties in the same way as the controller signal (pump). This necessitates weak probes and very small pump-to-probe gains (e.g. the fiber neurons in Hill et al. (2002); Rosenbluth et al. (2009)). Optical resonators can be used to strengthen these nonlinear effects at the cost of imposing wavelength constraints. In some approaches, output wavelength must be different than the input (e.g. with Kerr Xu and Lipson (2007), with XGM Barbay et al. (2011); Nahmias et al. (2013); Deng et al. (2017)), in which cases, one gate cannot drive another.

In some recent cases, cascadability is used to refer to a single feedforward connection Vaerenbergh et al. (2012); Van Vaerenbergh et al. (2013); Deng et al. (2017); Xiang et al. (2017), but we argue that this approach does not conclusively demonstrate indefinite physical cascadability. The format of the upstream input can be different than that of the downstream output such that the second might not be able to drive the first. The autapse provides a stronger demonstration of indefinite cascadability because its output and input signals are the same, and the upstream and downstream neurons are the same by definition. The recurrent circuit is equivalent to a neuron driving an indefinite feedforward chain of other identical neurons.

Some have approached cascadability by avoiding all-optoelectronic signal pathways and instead using an O/E/O chain consisting of a photodetector connected to a laser Nahmias et al. (2016); Shainline et al. (2018a, b) or modulator Tait et al. (2017); Nozaki et al. (2018). Wavelength constraints and phase sensitivity vanish because this information is lost in the electronic domain. The E/O conversion step can offer strong nonlinearity, as employed here, and the electronic domain itself offers efficient mechanisms for nonlinearity and amplification. In Shainline et al. (2017), an O/E/O neuron based on cryogenic silicon LEDs, superconducting detectors, and superconducting amplifiers McCaughan and Berggren (2014) was proposed. Its physical cascadability was demonstrated by the E/O/E LED-detector link shown in Buckley et al. (2017), and its gain cascadability has been addressed in more recent theoretical works Shainline et al. (2018a, b). The downside of O/E/O is a vulnerability to electrical parasitics; however, these parasitics can remain small regardless of network scale because O/E/O occurs entirely within a neuron, not between neurons.

When light combines, it interferes, posing a fundamental challenge to fan-in Goodman (1985). Optical fan-in results in either phase-dependence, when coherent, or $N$-fold loss, when incoherent (e.g. 3dB at $N=2$). In some all-optical devices where the in/out wavelengths can be the same (cascadable), these wavelengths also must be the same, meaning they cannot have more than one input Vaerenbergh et al. (2012); Van Vaerenbergh et al. (2013). Fan-in with coherent signals can be achieved by exerting complete control of the optical phase in the interconnect Shen et al. (2017), but then signal-dependent phase changes in a neuron profoundly affect the behavior of the subsequent interconnect, precluding any cascadability. In Shen et al. (2017), neuron calculations were implemented at low-speed in a CPU; a neuron based on saturable absorption was contemplated, but it was not discussed how this element would regenerate a consistent optical phase.

Fan-in has also been achieved using inputs that are coherent, but mutually incoherent, such as different spatial modes McCormick et al. (1993); Fok et al. (2011), different polarizations, or different wavelengths Khan et al. (2010); Chang et al. (2014); Tait et al. (2015). These signals do not interfere, and, since they are individually coherent, can be multiplexed and routed/weighted independently by tunable resonators Tait et al. (2016b); Xu and Soref (2011). Total power is sensed by a photodetector (O/E), making this fan-in approach compatible with the O/E/O approach to cascadability. Multi-wavelength weighted addition was combined with O/E/O laser neurons in Nahmias et al. (2016); Peng et al. (2018), wherein cascadability was also considered but not directly demonstrated. A downside of relying on multiple wavelengths is the need for a different laser source for each channel. The size of a single all-to-all subnetwork is capped by the available spectrum and the ability to distinguish adjacent channels, found in Tait et al. (2018d) to be less than 950 if using the resonators of Soltani et al. (2010); however, multiple of these subnetworks could be interfaced on a single chip Tait et al. (2014).

IV.2 Non-spiking photonic neurons

The great majority of work on photonic neurons has focused on lasers that implement spiking models similar to biological neurons Nahmias et al. (2013); Coomans et al. (2011); Van Vaerenbergh et al. (2013); Brunstein et al. (2012); Selmi et al. (2014); Romeira et al. (2016); Nahmias et al. (2016); Deng et al. (2017); Peng et al. (2018), reviewed in Prucnal et al. (2016). To claim a non-spiking modulator as a photonic neuron represents a departure from the origins of neuromorphic photonics. Modulator-class photonic neurons are easier to fabricate and dissipate less power on-chip. On the other hand, the processing repertoire of spiking neurons is theoretically greater than continuous-valued neurons.

The question is whether a move to continuous-valued photonic neurons would forfeit too much processing richness. Spiking networks can perform any continuous-valued task by representing values as mean firing rate (MFR). These are converted to analog signals by low-pass filtering. A mathematical derivation of the MFR transform is found in Tang et al. (2017). In many applications, a valid alternative is to use physically analog signals to represent the MFR directly. There is no debate over whether continuous-valued neurons have numerous engineering applications. For example, in traditional artificial neuromorphic computers, the spike rate is represented by analog voltage Rosenblatt (1958); Von Neumann (1956). Modern machine learning almost universally makes the MFR transform, representing MFR as floating-point numbers. Another example is the NEF compiling process Stewart and Eliasmith (2014) that only looks at MFR-to-MFR transfer functions. By replacing the MFR transfer functions with analog optical-to-optical transfer functions, we found that the whole NEF framework is still applicable to networks of modulator neurons Tait et al. (2017).

There are at least four situations where spiking artificial neurons have essential motivation; without these motivations, continuous-valued neurons are just as applicable. These motives include 1) studying brain function Friedmann et al. (2013); Diamond et al. (2016), 2) a need for increased robustness to amplitude noise, and 3) the use or study of temporal coding. Temporal coding has been proposed as one of the keys to energy efficiency in biological and perhaps artificial neural networks Sarpeshkar (1998); however, this proposition is debated and the subject of active theoretical research (generally originating from Maass (1997)). Finally, a need for spiking can stem from 4) hardware constraints. The modern wave of electronic neuromorphic hardware is based exclusively on spiking neural models because metal wires perform poorly when implementing dense, analog interconnects. Digital packet routing is used to multiplex a small number of physical interconnects, creating a large number of virtual interconnects Furber et al. (2014); Akopyan et al. (2015). In these cases, packets are treated as spikes.

There is no fundamental reason why photonics must have one of these motivations, so we believe that continuous-valued modulator neurons do not sacrifice too much computational richness. We postulate that, due to their complementary nature, laser and modulator neurons will come to address complementary application domains. It is even possible that they could exist on the same chip to handle different types of tasks within a single neural network.

IV.3 Caveats of the fiber-based autapse

Both here and in Tait et al. (2017), the feedback loop used for an autapse partially consists of fiber, resulting in substantial gain, frequency, and robustness discrepancies between the experiment and a fully integrated autapse. The high insertion loss of fiber-to-chip couplers necessitate an EDFA in the feedback loop, weakening the claim that we have directly demonstrated gain cascadability. Direct characterization showed a gain of 1.36, accounting for fiber-to-chip insertion loss (Sec. II.2); device-level calculations indicate that, without fiber-to-chip insertion loss, $g$ can readily exceed unity.

The kHz frequencies of the input sawtooths in Fig. 9(c-g) are chosen to isolate mono/bi-stable equilibrium effects from dynamical effects. If the frequency were too high, then the long fiber feedback delay would lead to obfuscating time-delayed dynamics Romeira et al. (2014). On the other hand, a modulation bandwidth that is too slow will also obscure equilibrium dynamics governed by carrier injection modulation. The thermal dissipation time constant is near 1ms, and the EDFA gain population timeconstant is 9ms. The temperature and population states effectively act as integrators around 1kHz and 100Hz, respectively. The resulting phase lag in the output closely resembles bistability. To eliminate the possibility that we are actually observing phase lag dynamics, two different frequencies (50kHz and 100kHz) are compared in Fig. 9(d vs. e and f vs. g). The fact that they produce very similar responses provides evidence that we are observing equilibrium effects, as opposed to dynamic effects that are frequency-dependent.

The fiber setup has poor environmental robustness. The couplers are polarization-sensitive, and the feedback fiber can drift. This means the polarization controller must be tuned relatively often to maintain a consistent round-trip gain. The average optical power coming out of the modulator is non-zero, thereby affecting the steady-state bias on the modulator. To further complicate that effect, the average power is weighted by the same amount as the varying signal (see Eq.(5) of Ref. Tait et al. (2016a)). To counter this effect, the prior work Tait et al. (2017) used an in-line capacitor as a DC block. That is not possible here because the key PD-modulator junction is fixed on-chip, so we perform this correction manually for the two feedback weight values. In a real network, it would be relatively straightforward to calculate the bias deflection due to weighted average power and then to correct the applied bias current to result in the desired net bias. We leave the automated version of that control algorithm to future work.

The fiber-based autapse has intriguing resemblances to photonic reservoir computers based on time-delayed feedback Larger et al. (2012); Hermans et al. (2015); Ortín et al. (2015); Soriano et al. (2015); Romeira et al. (2016) although they represent distinct approaches to computing. In fiber reservoirs, the feedback fiber is used to induce complex dynamics, unlike autapse demonstrations in which we attempt to minimize these dynamics in order to show isomorphism to a simple model. The reservoir concept complements neural network approaches, as they are typically understood, precisely because it does not rely on maintaining a correspondence between hardware and a theoretical model. This means reservoirs can employ continuous substrates that are easy to fabricate but difficult or impossible to make adhere to a model, such as optical phase around a fiber or wave amplitude in a bucket of water Fernando and Sojakka (2003). A network of neurons can also, of course, act as a reservoir by remaining ignorant to its theoretical model Maass et al. (2002). Reservoir computers rely on instance-specific training to isolate dynamics desirable for processing – there is no guarantee that a particular hardware instance will exhibit the desired dynamics at all. In contrast, any artificial system isomorphic to an abstract neural network is guaranteed to be capable of all dynamics possible in that model.

IV.4 Further work

The results of this paper pose questions for further research. It is apparent how the modulator neuron introduced in this work could be co-integrated with the microring weight of Ref. Tait et al. (2017), together shown in Fig. 1. An important demonstration for future work will be a fully-integrated autapse. An integrated autapse would not suffer from fiber-induced latency or fiber-to-chip coupling loss. As hypothesized in Tait (2018), an integrated autapse could provide a means to accurately and experimentally quantify the energy consumption of neuromorphic photonic systems.

Further work should look to reproduce these results at higher bandwidth. There were two bandwidth limiters: 1) the use of carrier injection modulation and 2) no isolation from the capacitance of the $I_{b}$ connection. The first limitation could be addressed by optimizing the MRR modulator for stronger depletion modulation. The second could be addressed by an on-chip resistor and series inductor. The capacitance of the modulator junction is affected by the metal between it and the dominant impedance of the biasing source. In this work, that included the junction itself, its breakout trace to the pad, the probe contacting the pad, and the wire leading to the Keithley. An on-chip resistor could constrain this capacitive region to just the junction itself.

Further work could also explore computational behavior by combining the experimentally validated transfer functions of the MRR modulator neuron with existing neural algorithms and compilers Stewart and Eliasmith (2014). These compilers take a neuron transfer function and a high-level task specification, then returning a weight matrix. By mapping these weights back to hardware, the physical neural network will perform the task. This is analogous to a program compiler that takes a computer architecture and a program written in a high-level language, then returning machine code. Prior work Tait et al. (2017) demonstrated this strategy in photonics using the Neural Engineering Framework (NEF) Eliasmith and Anderson (2004). Likewise, the validated transfer functions could be injected into algorithms for machine learning George et al. (2018).

V Conclusion

We fabricated a silicon photonic circuit consisting of two photodetectors electrically connected to a microring modulator. We then directly demonstrated its ability to act as a network-compatible photonic neuron, namely the properties of optical-to-optical nonlinearity, fan-in, and indefinite cascadability. These properties have never been demonstrated together in an integrated optoelectronic device.

Combined with microring weight banks shown in Tait et al. (2017, 2018a), the modulator neuron constitutes the final component needed to implement broadcast-and-weight Tait et al. (2014) neural networks: to date, the only architecture to propose a way to unite well-defined photonic neurons and well-defined neural interconnects upon a mainstream silicon photonic platform. Compatibility with silicon photonics provides crucial aspects of feasibility, scalability, and economies of scale to modern day photonic neural networks. Due to their unprecedented potential speed, neuromorphic photonic systems can come to bear on society’s growing demand for datacenter machine learning and could, additionally, open up unexplored regimes of ultrafast machine intelligence.

VI Appendix: equivalence of cascadability and bifurcation

Proving that particular piece of silicon exhibits a mathematical isomorphism to a desired model is not trivial, even for a system as simple as a single neuron. This proof is important because, if and only if a device is isomorphic to a model, can that model be used to predict behaviors of larger networks of those devices. Our approach to a rigorous photonic neural network demonstration is to derive qualitative predictions of a characteristic model, and then to observe them in experiment. This observability is crucial for connecting abstraction to reality. In Sec. III.5, we showed a mono/bi-stable bifurcation in an autapse in attempts to demonstrate cascadability. Here, we derive the equivalence between the cascadability condition and a bifurcation threshold, only the latter of which is an observable phenomenon.

The modulator neuron autapse is shown in Fig. 10. We assume that a voltage-mode depletion modulation effect is used instead of current-biased injection modulation as above because it will eventually be the preferred approach. The cascadability condition is met when the neuron’s differential optical-to-optical gain, $g$, exceeds unity.

VI.1 Theory

$g$ can be derived from device properties. It is defined {IEEEeqnarray}rCl g &= dPoutdPin

= dPoutdTmod dTmoddV dVdPin where $T_{mod}$ is modulator transmission, and $V$ is the junction voltage.

The power leaving the modulator is {IEEEeqnarray}rClP_out &= P_pump T_mod(V)

dPoutdTmod = P_pump where $P_{pump}$ is the optical power entering the modulator, and $V$ is the junction voltage. We assume that the modulator is biased at its maximum slope point for optimal modulation slope efficiency. When there are no inputs, $V=V_{b}$, where $V_{b}$ is the bias voltage. The optimal bias voltage is $V_{b}^{*}$. {IEEEeqnarray}rClV_b^* &= argmax_V dTmod(V)dV

dTmoddV—_V_b = V_b^* = π2 Vπ where $V_{\pi}$ is called the $\pi$-voltage, used for notational convenience. $V_{\pi}$ is inversely proportional to $dT_{mod}/dV$. It means the voltage to induce a $\pi$ shift in the Mach-Zehnder modulator with equivalent modulation slope efficiency, not the voltage to induce a $\pi$ shift around the MRR.

The detector and receiver component is {IEEEeqnarray}rClV &= V_b + R_pd R_b P_in

dVdPin = R_pd R_b where $R_{pd}$ is detector responsivity and $R_{b}$ is the biasing impedance. Combining the above terms, {IEEEeqnarray}rClg &= P_pump ⋅π2 Vπ ⋅R_pd R_b

P_pump—_g=1 = 2 Vπ**πRpdRb This pump power is the minimum needed for cascadability.

VI.2 Observation

The physical autapse circuit can be modeled as a dynamical system whose state variable is the modulator voltage. A bifurcation is a qualitative change that occurs when the number or stability of fixed points changes. In the Supplementary Material of Tait et al. (2017), the fixed points were derived with a simplified model. Here, we examine fixed point stability in a physical model and find the bifurcation point to be identical to the expression in Eq. VI.1.

The modulator junction has these dynamics {IEEEeqnarray}rCldVdt &= - V - VbRbCmod + ipdCmod where $C_{mod}$ is the capacitance of the modulator junction, and $i_{pd}$ is the photocurrent. Photocurrent is induced by either the external signal, $P_{ext}$, or the self-feedback signal, $P_{in}$. Due to the feedback, $P_{in}=P_{out}$. {IEEEeqnarray}rCli_pd &= R_pd [P_out(V) + P_ext]

Zeroing the external input, this results in {IEEEeqnarray}rCldVdt &= - V - VbRbCmod + RpdCmod P_pump T_mod(V) We do not seek the solutions for the steady-state values because they have an inelegant dependence on the parameters. Instead, we assess the stability using the Jacobian linearization. In general, dynamical system are described by $\dot{x}=\mathbf{F}(x)$, where $\mathbf{F}$ is a matrix of functions. They can be linearized around fixed points, $x_{fp}$, as {IEEEeqnarray}rCllim_Δx →0 Δ˙x &= J Δx

where  J ≡ ∂F∂x—_x_fp where $\Delta x=x-x_{fp}$ and $\mathbf{J}$ is called the Jacobian matrix. In our one-dimensional case, the Jacobian is just a scalar, $J$. When it is positive $\Delta x$ will grow exponentially, meaning that the fixed point is unstable. A bifurcation occurs when stability changes, meaning when $J=0$. Of course, the exponential growth saturates at one of two other fixed points. In other words, in a 1-dimensional system, this bifurcation is a transition from monostability to bistability.

Using the physical description of the autapse dynamical system from Eq. (VI.2), {IEEEeqnarray}rClJ &= - 1RbCmod + RpdCmod P_pump dTdV—_V_b^* which crosses zero when {IEEEeqnarray}rClP_pump—_J=0 &= 2 Vπ**πRpdRb Thus, the expression for pump power where the autapse loses monostability corresponds exactly with that where the cascadability condition is met in Eq. (VI.1).

Funding

National Science Foundation (NSF) (ECCS 1247298, DGE 1148900)

Acknowledgment

Devices were fabricated at the IME A∗STAR foundry in Singapore. Fabrication support was provided via the Natural Sciences and Engineering Research Council of Canada (NSERC) Silicon Electronic-Photonic Integrated Circuits (SiEPIC) Program and the Canadian Microelectronics Corporation (CMC).