arXiv:2509.24266v2 Announce Type: replace 
Abstract: Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence, but their continued scaling poses challenges for resource-limited deployment. Despite recent advances in binary SNNs, the storage and computational demands remain substantial for large-scale networks. To further explore the compression and acceleration potential of SNNs, we propose Sub-bit Spiking Neural Networks (S$^2$NNs) that represent weights with less than one bit. Specifically, we first establish an S$^2$NN baseline by leveraging the clustering patterns of kernels in well-trained binary SNNs. This baseline is highly efficient but suffers from \textit{outlier-induced codeword selection bias} during training. To mitigate this issue, we propose an \textit{outlier-aware sub-bit weight quantization} (OS-Quant) method, which optimizes codeword selection by identifying and adaptively scaling outliers. Furthermore, we propose a \textit{membrane potential-based feature distillation} (MPFD) method, improving the performance of highly compressed S$^2$NN via more precise guidance from a teacher model. Extensive results on vision tasks reveal that S$^2$NN outperforms existing quantized SNNs in both performance and efficiency, making it promising for edge computing applications.

قدم الباحثون شبكات الأعصاب النبضية الفرعية (S$^2$NN)، وهو تقدم واعد في مجال الذكاء الاصطناعي. تهدف هذه الشبكات إلى معالجة تحديات قيود الموارد من خلال تقديم طريقة أكثر كفاءة في استخدام الطاقة لتوسيع نطاق الشبكات العصبية النبضية (SNN). من خلال تمثيل الأوزان بعدد أقل من البتات، يمكن أن تقلل S$^2$NN بشكل كبير من متطلبات التخزين والحوسبة، مما يسهل نشر الشبكات على نطاق واسع. هذه الابتكار مهم لأنه يمهد الطريق لتقنيات ذكاء اصطناعي أكثر سهولة وكفاءة.

Investigadores han presentado las Redes Neuronales de Espiga Sub-bit (S$^2$NN), un avance prometedora en el campo de la inteligencia artificial. Estas redes buscan abordar los desafíos de las limitaciones de recursos al ofrecer una forma más eficiente en energía de escalar las Redes Neuronales de Espiga (SNN). Al representar los pesos con menos bits, las S$^2$NN podrían reducir significativamente las demandas de almacenamiento y computación, facilitando el despliegue de redes a gran escala. Esta innovación es crucial, ya que allana el camino para tecnologías de IA más accesibles y eficientes.

Des chercheurs ont présenté les réseaux neuronaux à impulsions sub-bit (S$^2$NN), une avancée prometteuse dans le domaine de l'intelligence artificielle. Ces réseaux visent à relever les défis des limitations de ressources en offrant une méthode plus économe en énergie pour l'échelle des réseaux neuronaux à impulsions (SNN). En représentant les poids avec moins de bits, les S$^2$NN pourraient réduire considérablement les besoins en stockage et en calcul, facilitant ainsi le déploiement de réseaux à grande échelle. Cette innovation est cruciale car elle ouvre la voie à des technologies d'IA plus accessibles et efficaces.

Researchers have introduced Sub-bit Spiking Neural Networks (S$^2$NNs), a promising advancement in the field of machine intelligence. These networks aim to address the challenges of resource limitations by offering a more energy-efficient way to scale Spiking Neural Networks (SNNs). By representing weights with fewer bits, S$^2$NNs could significantly reduce storage and computational demands, making it easier to deploy large-scale networks. This innovation is crucial as it paves the way for more accessible and efficient AI technologies.

S$^2$NN: Sub-bit Spiking Neural Networks

arXiv:2601.08447v1 Announce Type: cross 
Abstract: Spike-timing-dependent plasticity (STDP) provides a biologically-plausible learning mechanism for spiking neural networks (SNNs); however, Hebbian weight updates in architectures with recurrent connections suffer from pathological weight dynamics: unbounded growth, catastrophic forgetting, and loss of representational diversity. We propose a neuromorphic regularization scheme inspired by the synaptic homeostasis hypothesis: periodic offline phases during which external inputs are suppressed, synaptic weights undergo stochastic decay toward a homeostatic baseline, and spontaneous activity enables memory consolidation. We demonstrate that this sleep-wake cycle prevents weight saturation while preserving learned structure. Empirically, we find that low to intermediate sleep durations (10-20\% of training) improve stability on MNIST-like benchmarks in our STDP-SNN model, without any data-specific hyperparameter tuning. In contrast, the same sleep intervention yields no measurable benefit for the surrogate-gradient spiking neural network (SG-SNN). Taken together, these results suggest that periodic, sleep-based renormalization may represent a fundamental mechanism for stabilizing local Hebbian learning in neuromorphic systems, while also indicating that special care is required when integrating such protocols with existing gradient-based optimization methods.

تشير دراسة جديدة إلى اقتراح نظام تنظيم منزلي قائم على النوم لاستقرار البلاستيك المعتمد على توقيت النبضات (STDP) في الشبكات العصبية النابضة المتكررة (SNNs). تهدف هذه الطريقة إلى التخفيف من مشكلات مثل النمو غير المحدود للأوزان والنسيان الكارثي من خلال إدخال مراحل غير نشطة حيث تتراجع الأوزان المشبكية نحو خط الأساس المنزلي، مما يعزز من دمج الذاكرة.

Un nuevo estudio propone un esquema de regularización homeostática basado en el sueño para estabilizar la plasticidad dependiente del tiempo de picos (STDP) en redes neuronales recurrentes de picos (SNNs). Este enfoque busca mitigar problemas como el crecimiento ilimitado de pesos y el olvido catastrófico al introducir fases fuera de línea donde los pesos sinápticos decaen hacia una línea base homeostática, mejorando así la consolidación de la memoria.

Une nouvelle étude propose un schéma de régularisation homéostatique basé sur le sommeil pour stabiliser la plasticité dépendante du timing des pics (STDP) dans les réseaux neuronaux à pics récurrents (SNNs). Cette approche vise à atténuer des problèmes tels que la croissance illimitée des poids et l'oubli catastrophique en introduisant des phases hors ligne où les poids synaptiques décroissent vers une ligne de base homéostatique, améliorant ainsi la consolidation de la mémoire.

A new study proposes a sleep-based homeostatic regularization scheme to stabilize spike-timing-dependent plasticity (STDP) in recurrent spiking neural networks (SNNs). This approach aims to mitigate issues such as unbounded weight growth and catastrophic forgetting by introducing offline phases where synaptic weights decay towards a homeostatic baseline, enhancing memory consolidation.

S$^2$NN: Sub-bit Spiking Neural Networks

Was this article worth reading? Share it

LucidQuery AI

Brainactive

Aqaba.ai

Superset

Snapshot AI

Subtle

Ready to build your own newsroom?