Spiking neural networks (SNNs) have emerged as a hardware efficient architecture for classification tasks.The challenge of spike-based encoding has been the lack of a universal training mechanism performed entirely using spikes.There have been several attempts to adopt the powerful backpropagation (BP) technique used in non-spiking artificial neural networks (ANNs): (1) SNNs can be trained by externally computed numerical gradients.(2) A major advancement towards native spike-based learning has been the use of approximate BP using spike-time dependent plasticity with phased forward/backward passes.
However, the transfer of information between such phases for gradient and weight update calculation necessitates external memory and computational access.This is a challenge for standard neuromorphic hardware implementations.In this paper, we propose a stochastic SNN based back-prop (SSNN-BP) algorithm that utilizes a composite neuron to simultaneously compute the forward pass activations and backward pass gradients explicitly Marcas culturais na escrita do gênero resenha em português e espanhol with spikes.Although signed gradient values are a challenge for spike-based representation, we tackle this by splitting the gradient signal into positive and negative streams.
The composite neuron encodes information in the form of stochastic spike-trains and converts BP weight updates into temporally and spatially local spike coincidence updates compatible with hardware-friendly resistive processing units.Furthermore, we characterize the quantization effect of discrete spike-based weight update to show that our method approaches BP ANN baseline with sufficiently long spike-trains.Finally, we show that the well-performing softmax cross-entropy loss function can be implemented through inhibitory lateral connections enforcing a winner take all rule.Our SNN with a two-layer network shows excellent generalization through comparable performance to ANNs with equivalent architecture and regularization parameters on static image datasets like MNIST, Fashion-MNIST, Extended MNIST, and temporally encoded image datasets like Neuromorphic MNIST datasets.
Thus, SSNN-BP enables BP compatible with purely Evaluation of nutrients in bread: a systematic review spike-based neuromorphic hardware.