arXiv:2505.18608v5 Announce Type: replace 
Abstract: Spiking Neural Networks promise brain-inspired and energy-efficient computation by transmitting information through binary (0/1) spikes. Yet, their performance still lags behind that of artificial neural networks, often assumed to result from information loss caused by sparse and binary activations. In this work, we challenge this long-standing assumption and reveal a previously overlooked frequency bias: spiking neurons inherently suppress high-frequency components and preferentially propagate low-frequency information. This frequency-domain imbalance, we argue, is the root cause of degraded feature representation in SNNs. Empirically, on Spiking Transformers, adopting Avg-Pooling (low-pass) for token mixing lowers performance to 76.73% on Cifar-100, whereas replacing it with Max-Pool (high-pass) pushes the top-1 accuracy to 79.12%. Accordingly, we introduce Max-Former that restores high-frequency signals through two frequency-enhancing operators: (1) extra Max-Pool in patch embedding, and (2) Depth-Wise Convolution in place of self-attention. Notably, Max-Former attains 82.39% top-1 accuracy on ImageNet using only 63.99M parameters, surpassing Spikformer (74.81%, 66.34M) by +7.58%. Extending our insight beyond transformers, our Max-ResNet-18 achieves state-of-the-art performance on convolution-based benchmarks: 97.17% on CIFAR-10 and 83.06% on CIFAR-100. We hope this simple yet effective solution inspires future research to explore the distinctive nature of spiking neural networks. Code is available: https://github.com/bic-L/MaxFormer.

تتحدى الأبحاث الحديثة الافتراض القائل بأن الشبكات العصبية النابضة (SNN) تعاني من أداء أقل مقارنة بالشبكات العصبية الاصطناعية (ANN) بسبب فقدان المعلومات الناتج عن التنشيطات النادرة. بدلاً من ذلك، تسلط الضوء على تحيز التردد حيث تقوم الخلايا العصبية النابضة بقمع المعلومات عالية التردد. هذه النتيجة مهمة لأنها قد تؤدي إلى تحسينات في تصميم وتطبيق SNN، مما يجعلها أكثر تنافسية في المهام الحسابية، خاصة في الحوسبة الموفرة للطاقة.

Una investigación reciente desafía la suposición de que las redes neuronales espinosas (SNN) tienen un rendimiento inferior en comparación con las redes neuronales artificiales (ANN) debido a la pérdida de información causada por activaciones escasas. En cambio, destaca un sesgo de frecuencia donde las neuronas espinosas suprimen la información de alta frecuencia. Este hallazgo es significativo, ya que podría llevar a mejoras en el diseño y la aplicación de las SNN, haciéndolas más competitivas en tareas computacionales, especialmente en computación eficiente en energía.

Une recherche récente remet en question l'idée que les réseaux de neurones à impulsion (SNN) sont moins performants que les réseaux de neurones artificiels (ANN) à cause de la perte d'information due à des activations rares. Au lieu de cela, elle met en lumière un biais de fréquence où les neurones à impulsion suppriment l'information à haute fréquence. Cette découverte est importante car elle pourrait conduire à des améliorations dans la conception et l'application des SNN, les rendant plus compétitifs dans les tâches de calcul, notamment dans l'informatique économe en énergie.

Recent research challenges the assumption that spiking neural networks (SNNs) underperform compared to artificial neural networks (ANNs) due to information loss from sparse activations. Instead, it highlights a frequency bias where spiking neurons suppress high-frequency information. This finding is significant as it could lead to improvements in the design and application of SNNs, making them more competitive in computational tasks, particularly in energy-efficient computing.

Spiking Neural Networks Need High Frequency Information

arXiv:2601.08447v1 Announce Type: cross 
Abstract: Spike-timing-dependent plasticity (STDP) provides a biologically-plausible learning mechanism for spiking neural networks (SNNs); however, Hebbian weight updates in architectures with recurrent connections suffer from pathological weight dynamics: unbounded growth, catastrophic forgetting, and loss of representational diversity. We propose a neuromorphic regularization scheme inspired by the synaptic homeostasis hypothesis: periodic offline phases during which external inputs are suppressed, synaptic weights undergo stochastic decay toward a homeostatic baseline, and spontaneous activity enables memory consolidation. We demonstrate that this sleep-wake cycle prevents weight saturation while preserving learned structure. Empirically, we find that low to intermediate sleep durations (10-20\% of training) improve stability on MNIST-like benchmarks in our STDP-SNN model, without any data-specific hyperparameter tuning. In contrast, the same sleep intervention yields no measurable benefit for the surrogate-gradient spiking neural network (SG-SNN). Taken together, these results suggest that periodic, sleep-based renormalization may represent a fundamental mechanism for stabilizing local Hebbian learning in neuromorphic systems, while also indicating that special care is required when integrating such protocols with existing gradient-based optimization methods.

تشير دراسة جديدة إلى اقتراح نظام تنظيم منزلي قائم على النوم لاستقرار البلاستيك المعتمد على توقيت النبضات (STDP) في الشبكات العصبية النابضة المتكررة (SNNs). تهدف هذه الطريقة إلى التخفيف من مشكلات مثل النمو غير المحدود للأوزان والنسيان الكارثي من خلال إدخال مراحل غير نشطة حيث تتراجع الأوزان المشبكية نحو خط الأساس المنزلي، مما يعزز من دمج الذاكرة.

Un nuevo estudio propone un esquema de regularización homeostática basado en el sueño para estabilizar la plasticidad dependiente del tiempo de picos (STDP) en redes neuronales recurrentes de picos (SNNs). Este enfoque busca mitigar problemas como el crecimiento ilimitado de pesos y el olvido catastrófico al introducir fases fuera de línea donde los pesos sinápticos decaen hacia una línea base homeostática, mejorando así la consolidación de la memoria.

Une nouvelle étude propose un schéma de régularisation homéostatique basé sur le sommeil pour stabiliser la plasticité dépendante du timing des pics (STDP) dans les réseaux neuronaux à pics récurrents (SNNs). Cette approche vise à atténuer des problèmes tels que la croissance illimitée des poids et l'oubli catastrophique en introduisant des phases hors ligne où les poids synaptiques décroissent vers une ligne de base homéostatique, améliorant ainsi la consolidation de la mémoire.

A new study proposes a sleep-based homeostatic regularization scheme to stabilize spike-timing-dependent plasticity (STDP) in recurrent spiking neural networks (SNNs). This approach aims to mitigate issues such as unbounded weight growth and catastrophic forgetting by introducing offline phases where synaptic weights decay towards a homeostatic baseline, enhancing memory consolidation.

Spiking Neural Networks Need High Frequency Information

Was this article worth reading? Share it

LucidQuery AI

Brainactive

Spicy Ranks AI

AINF Membership

Bytefitz

Graphite Note

Ready to build your own newsroom?