arXiv:2511.08092v1 Announce Type: cross 
Abstract: We challenge the conventional view of neural network pruning as solely a compression technique, demonstrating that one-shot magnitude pruning serves as a powerful implicit regularizer for ASR. Using Whisper-small, we combine gradient- and Fisher-based sensitivity diagnostics with targeted, component-wise pruning. This reveals architectural asymmetries: decoder FFNs are pruning-fragile, whereas decoder self-attention and the last encoder layers contain redundancy that, when removed, improves generalization. Without fine-tuning, pruning 50% of decoder self-attention reduces WER by 2.38% absolute (20.44% relative) on LibriSpeech test-other; pruning the last four encoder layers at 50% instead yields a 1.72% absolute (14.8% relative) improvement. Gains persisted on Common Voice and TED-LIUM datasets. Beyond regularization benefits, our sensitivity-aware approach enables more aggressive one-shot compression. At 40% sparsity, where established global pruning approaches catastrophically fail, our method preserves near-baseline accuracy. This positions pruning as a first-class architectural design tool: knowing where to prune is as important as how much to prune.

A recent study published on arXiv demonstrates that one-shot magnitude pruning can serve as an effective regularization technique in automatic speech recognition (ASR). By applying sensitivity-aware pruning to the Whisper-small model, researchers found that pruning 50% of decoder self-attention layers improved word error rates significantly. This approach not only enhances model performance but also positions pruning as a vital architectural design tool, allowing for more aggressive compression without sacrificing accuracy.

Pruning as Regularization: Sensitivity-Aware One-Shot Pruning in ASR

Was this article worth reading? Share it

Ready to build your own newsroom?