MPD-SGR: Robust Spiking Neural Networks with Membrane Potential Distribution-Driven Surrogate Gradient Regularization

Minimizing inconsistencies across successive versions of an AI system is crucial in image classification, particularly as the number of training classes increases. Negative flips occur when an updated model misclassifies previously correctly classified samples. This issue intensifies with the addition of new categories, which can reduce the margin of each class and introduce conflicting patterns. A novel approach is proposed to preserve the margins of the original model while improving performance, encouraging a larger relative margin between learned and new classes.

The paper titled 'StochEP: Stochastic Equilibrium Propagation for Spiking Convergent Recurrent Neural Networks' introduces a new framework for training Spiking Neural Networks (SNNs) using Stochastic Equilibrium Propagation (EP). This method aims to enhance training stability and scalability by integrating probabilistic spiking neurons, addressing limitations of traditional Backpropagation Through Time (BPTT) and deterministic EP approaches. The proposed framework shows promise in narrowing performance gaps in vision benchmarks.

Spiking Neural Networks (SNNs) are gaining popularity due to their energy efficiency, but they face challenges in effective training. Recent advancements have introduced knowledge distillation (KD) techniques, utilizing pre-trained artificial neural networks (ANNs) as teachers for SNNs. This process typically aligns features and predictions from both networks, but often overlooks their architectural differences. To address this, two new KD strategies, Saliency-scaled Activation Map Distillation (SAMD) and Noise-smoothed Logits Distillation (NLD), have been proposed to enhance training effectiv…