Pruning as Regularization: Sensitivity-Aware One-Shot Pruning in ASR

The publication of the study on arXiv highlights a shift in understanding neural network pruning, particularly in the context of automatic speech recognition (ASR). Traditionally viewed as a mere compression technique, the research reveals that one-shot magnitude pruning can act as a powerful implicit regularizer. By utilizing the Whisper-small model, the team combined gradient- and Fisher-based sensitivity diagnostics with targeted pruning, uncovering that decoder feedforward networks (FFNs) are particularly fragile to pruning. This insight led to significant improvements in generalization, with pruning 50% of decoder self-attention yielding a 2.38% absolute reduction in word error rates on the LibriSpeech test set. Additionally, pruning the last four encoder layers at the same rate resulted in a 1.72% absolute improvement. The gains were consistent across other datasets like Common Voice and TED-LIUM. Notably, the sensitivity-aware approach allows for maintaining near-baseline accura…