1.

INTRODUCTION

Recurrent neural networks are widely used to address sequential data analysis problems and exhibit internal memory. Within this broad category, the architecture referred to as Echo State Network is especially appealing for photonic implementations and the realization of high speed, low power analog devices for accelerator and edge computing applications. The network is composed of an input layer, an internal node structure called a Reservoir, and an Output layer. The interconnections between the neurons inside the reservoir, as well as those between the input layer and the reservoir, are fixed and randomly assigned with a density adjusted to optimize performance by creating conditions such that the system is at the edge of stability. Only the inteconnections between the reservoir and the output layers are adjusted to perform a determined task. A major benefit is that this training procedure is a linear regression, i.e., may be performed by a simple matrix inversion without the need for a backpropagation procedure. Such a system is depicted schematically in Fig. 1(a).

Figure 1.

(a) Architecture of a reservoir neural network showing the Input, Reservoir and Output layers. (b) Implementation of a reservoir computer using a physical medium with some complex internal dynamics. (c) Reservoir implementation in which a delayed feedback and time multiplexing scheme are used with virtual nodes.

Even though this architecture was initially proposed before the advent of Deep Learning, it remains popular today in contexts where stable and simple training is preferred.¹ Moreover, as the internal structure of the network does not have to be precisely determined, reservoir computing is appealing for hardware implementations with a physical medium exhibiting complex behavior as shown in Fig. 1(b). So far, although systems have been realized with a water bucket, integrated circuits, memristive devices, spintronic materials, springs, and biological media, publications on photonic implementations have been especially numerous and include designs based on waveguides on a chip, photonic crystals and free-space optics.²

A related advantage of this architecture is the possibility to use for computation, instead of a physical node array, a single time-varying neuron with a delayed feedback loop and an encoded input to obtain a time-multiplexing scheme as shown in Fig. 1(c). Even though this scheme was originally demonstrated with an electronic circuit,³ several implementations with an optical feedback have been reported as shown in Fig. 2. In those systems, the loop may undego an electronic conversion for the readout and the combination with the input. An all-optical design has also been reported, with an optical amplifier. An alternative is an opto-electronic conversion inside the neuron to provide gain, with the signal delay, splitting and combining (fan-out and fan-in) functions performed optically. Alternatively, several authors have reported on a delayed optical feedback re-entering a laser cavity providing rich non-linearity to realize a reservoir system. Most of these original reports were table-top demonstrations using a single channel fiber in the hundreds of meter range. An improvement of this technology would be an integration on a board to obtain a compact device with several hundred nodes.

Figure 2.

Various photonic implementations of time-delay reservoirs. (a) Electronic conversion in the feedback loop and combination of the input and feedback signals. (b) All-optical feedback loop with gain. (c) Opto-electronic conversion inside the node. (d) Optical feedback into a laser cavity.

Another application in which photonics has had a large impact in the last decade is Optical Interconnects, which enables very low power and high channel density data transfer within data centers and high performance computers. Within a system, this technology can be used for core-to-core, chip-to-chip, board-to-board or rack- to-rack communication spanning a range from the centimeter to the hundreds of meter scale. Today, the preferred configuration to respond to this need is the use of VCSEL and photodiode chip arrays transmitting and receiving light via multi-mode waveguides. Fig. (3) shows a polymer waveguide-based optical multi-chip module for this purpose.⁴ A CPU is electrically linked to a laser driver, transforming CMOS-compatible digital data into a signal with a bias and an amplitude large enough to modulate the lasers, which are flip-chip mounted on a board. Light is reflected by micro-mirrors and is coupled into waveguide cores having a typical size of 30-50 um. A ribbon fiber may or may not be inserted in the light path. After reflection on a second micro-mirror, photodiode and trans-impedance limiting amplifiers chip arrays convert back the data to be compatible with a second CPU. Several parameters must be adjusted to minimize power consumption, depending on the speed and loss in the optical path.

Figure 3.

Polymer waveguide-based optical interconnect modules for high speed, low power, high channel density data transfer in a high performance computer. The lower part shows the various parameters that can be tuned to optimize the system.

In this work, our aim is to describe preliminary results towards the realization of a time-delay reservoir computing cluster on a board with VCSEL and photodiode chip arrays for high speed, low power data processing in a compact format.

2.

SYSTEM DESIGN

In the system that we design shown in Fig. (4), the neuron is formed by a photodiode connected back-to-back to a VCSEL with suitable amplification in-between. The input symbols with the modulation mask are programmed into an FPGA driving a first VCSEL. A graded index multi-mode optical fiber transmits the signal to the neuron. After opto-electronic conversion, the emitted light is split by fiber couplers into a component that goes to the delay loop to be combined with the input, and another component that reaches a second photodiode connected to a time sampling oscilloscope for off-line processing and training. In the final design of this concept, all components will be integrated on a board with polymer waveguides. All twelve channels will be operational and interconnected to obtain several virtual and physical nodes. Several arrays may also be cascaded to form a rectangular set of neurons interconnected with waveguides that may have crossing patterns.

Figure 4.

Reservoir system built by linking back-to-back photodiode and laser chips. An amplifier in between provides non-linearity. Lines in blue indicate multimode fiber splitter and combiner. Coupling between the opto-chips and the fibers is made using positionners and multi-mode lens fibers (Cascade Microtech).

The components required to build such a system, including micro-mirrors, channel crossings and compact optical delay lines, have already been demonstrated at a research level. The only element that will have to be realized is a twelve-channel amplifier array driving the neuron VCSELs, shown in Fig. 5(a). The amplifiers should have a gain that is not too large to reproduce an analog waveform, as well as some saturation to provide a non-linearity improving performance (Fig. 5(b)). Moreover the bandwidth is important, as a response somewhat slower than the incoming signal modulation will provide some desired interaction between nodes. We analyze the system assuming 3-dB bandwidth cutoff varying between 1 and 100 GHz (Fig. 5(c)).

Figure 5.

(a) Enlarged view of a twelve-neuron array in the proposed design. (b) Designed gain characteristics of an amplifier. The red line shows a straight line to emphasize saturation in the actual gain curve in blue. (c) Three possible bandwidth properties for an amplifier, going from a relatively low-spec case of 1 GHz to a very high speed device with 100 GHz.

For the system simulation, it is important to reproduce the experimental lineshape of the neurons as a function of the component parameters for conversion into an accurate matrix formalism to estimate reservoir performance. The node response (corresponding to the optical output power of the neuron) is given by

where F_NL is the nonlinear function of the neuron, u(n) is the serial signal input, m_i is the scaling step mask randomly chosen in the range [-1:1] and kept fixed for each symbol, and x_i(n) is the physical node state for virtual node i at symbol n.

The laser dynamics is related to the photon and electron densities S_j and N_j in the resonant cavity at a time step j (each virtual node step i is divided into j_max smaller time steps for the finite difference calculation). They are related by the equations

where τ_ph, τ_n, β, Γ, v_g, V and η_j are the photon and carrier lifetimes, the spontaneous emission factor, the confinement factor, the group velocity, the cavity volume and the slope efficiency respectively. G_j is given by

where g₀, N_tr and ϵ are the cavity gain, transparent carrier density and the gain compression factor respectively. The optical output power is proportional to the photon density in the cavity, multiplied by the coupling loss factor to obtain the light power into the waveguide. For the photodiode, we assume a 0.7 A/W responsivity with a response time that is fast enough to have a negligible effect on the signal.

To provide non-linearity to the system and protect the second VCSEL from a signal surge, the limiting amplifier configuration with a voltage output v_out that we choose is described by the Cann model⁵ as

where g is the linear gain and v_in is the output of the 50 Ω transimpedance component. The resulting curve is shown in Fig. 5(b), with chosen parameters L = 0.8 and s = 2.0. The high speed characteristic is adjusted by applying a third order Butterworth filter and adjusting the cutoff frequency. Moreover, the DC signal is not transmitted by the amplifier. The optical ouptut from the first VCSEL is calculated in a similar manner, using an electrical excitation including a lowpass filter reproducing the actual FPGA output.

A typical full lineshape used for training a reservoir may contain several thousand symbols and hundreds of virtual nodes, with each virtual node step containing several data points for a precise simulation. To take into account the frequency response of the amplifier, it is advantageous to perform part of the computation in the frequency domain. However, the total number of data points is typically too large to perform a Fast Fourier transform. Moveover, the feedback component must be included in the input. For these reasons, we adopt a staggered computation method. Several symbol steps, along with an enlarged view, are shown in Fig. 6(a) and (b) respectively. For each step, half step paddings at the front and back of the data are included to compute the amplifier response in the frequency domain. Using appropriate boundary conditions, the node VCSEL lineshape, signal output and feedback for the next symbol step are then computed in the time domain from the laser equation set described above. This procedure is repeated for every symbol step. Fig. 6(c) shows the lineshape after input mask modulation. Each mask step is divided into 64 data points for the numerical simulation.

Figure 6.

(a) Part of a distorted random symbol stream used as reservoir input. (b) Enlarged view showing staggered pattern for fast fourier transform calculation in the frequency domain. (c) Symbol pattern modulated by the input mask for the virtual nodes. Each symbol step is multiplied by the same mask.

The amplifier chip has a 50 Ω input impedance, and the power gain to drive the second VCSEL is set to 19 dB. Fig. 7(a) shows the calculated echo response to a step input in our design. The feedback pulse amplitude decreases to approximately one third of the previous step, a condition for which we obtain good results for a signal recovery task. Fig. 7(b) and (c) show experimental results of a step input and system delayed feedback, in good agreement with the calculations. A higher gain results in uncontrolled resonance of the reservoir, making it unusable.

Figure 7.

(a) Calculated reservoir ouptut for a single step symmetric input pulse. (b) Experimental input pulse from the FPGA. (c) Experimental reservoir output signal with the input shown in (b) showing the effect of delayed feedback. (The time scales are different for the experimental and calculated results due to different input data rate and feedback delay, but the amplitude variation is similar.)

3.

NUMERICAL AND EXPERIMENTAL RESULTS

The task that we select for this demonstration is the recovery of a distorted signal incoming on a detector several times after multiple reflections with added noise. A series of random integers d in the range {-3,-1,1,3} is generated for a number of steps n_max. The distorted signal u(n) having non-linear and inter-symbol interference components is computed according to the relation

where ν(n) is a random noise with a Gaussian distribution that we adjust to obtain a relatively low signal-to-noise ratio of 20 dB. This signal is then normalized to obtain a suitable voltage input into the VCSEL.

We first perform a simulation of the reservoir performance assuming that each virtual node step has a 40 ps time duration, i.e., a 25 GS/s input rate, with 50 virtual nodes. The mask is chosen randomly in the interval [-1:1] and repeatedly applied to each input symbol. The 3-dB bandwidth of the amplifier is varied between 1 and 100 GHz. A set of 2000 symbols is used for training, and for the performance evaluation five datasets with 1000 symbols each are used. As shown in Fig. 8(a), the error range remains slightly above 10⁻⁴ in every test and performance of the system does not show any clear trend with the speed of the amplifier for this calculation. Fig. 8(b) and (c) show the signal lineshape for the minimum and maximum amplifier speed respectively. It is clear that in the first case, the system response does not follow the input node pattern. In real experimental system, gaussian noise would be present and would prevent good operation in this range. For the high speed case however, the node response is fast enough to carry the full node lineshape. Such conditions are promising for an experimental implementation of this fast signal.

Figure 8.

(a) Calculated error ratio of the reservoir system for different bandwidths of the amplifier in the node. Input (blue) and Output (red) lineshapes of the reservoir system for 1 GHz (b) and 100 GHz (c) amplifier bandiwdths.

We now discuss the design and performance of the experimental reservoir that we built. The amplifier is and SHF 450B with a 40 GHz bandwidth. The signal rate determined by the FPGA is 1 GS/s and the feedback loop length is approximately 4m, which results in i_max = 25 virtual nodes. The time duration of each mask step m_i is thus 1 ns. The input and output attenuation factors are determined by the optical loss in the fiber splitter, combiner and couplers as well as the gain in the neuron, and are adjusted experimentally to remain close to the edge of the stability. Fig. 9(a) shows a representative part of the input pattern programmed into the FPGA, while Fig. 9(b) shows the measured eletrical output of the FPGA. Fig. 9(c) shows the reservoir output after detection by the second photodiode. As expected, the shape is related to the input but not identical, due to the memory effect of the system. Fig. 9(d) shows the performance of the experimental system. Six data sets with 800 symbols each were recorded, one for training and five for testing. The number of errors is higher than for the calculated result, but still low. A detailed analysis of the causes of errors is currently in progress.

Figure 9.

Calculated input (a), measured input (b) and measured output (c) lineshapes of the system for a small time interval. (d) Experimental and calculated error count for the five test data batches.

4.

CONCLUSION

In conclusion, we describe a platform for the realisation of a photonic reservoir cluster on a board using VCSEL and photodiode chip arrays, polymer waveguides and micro-mirrors. A simulation framework taking into account laser cavity dynamics, components bandwidth, electrical gain, and optical loss for the performance assessment of a reservoir computing device is described. A fiber-based experimental system was built and operated to perform a distorted signal recovery task. Results are promising for the realization of a reservoir cluster device on a board operating at high speed with several hundreds of nodes.