Stabilizing Direct Training of Spiking Neural Networks: Membrane Potential Initialization and Threshold-robust Surrogate Gradient

The recent paper on stabilizing the direct training of Spiking Neural Networks (SNNs) presents two key innovations: Membrane Potential Initialization (MP-Init) and Threshold-robust Surrogate Gradient (TrSG). These advancements tackle persistent challenges in SNNs, such as temporal covariate shift (TCS) and unstable gradient flow, which hinder effective training. By aligning the initial membrane potential with its stationary distribution, MP-Init mitigates TCS, while TrSG stabilizes gradient flow concerning neuron thresholds. Extensive experiments validate these methods, demonstrating state-of-the-art accuracy on both static and dynamic image datasets. This research not only enhances the performance of SNNs but also paves the way for novel energy-efficient AI paradigms, marking a significant step forward in the field of artificial intelligence.