Alignment-Constrained Dynamic Pruning for LLMs: Identifying and Preserving Alignment-Critical Circuits

The introduction of Alignment-Aware Probe Pruning (AAPP) marks a significant advancement in the deployment of Large Language Models (LLMs). As LLMs require substantial computational resources, dynamic pruning has emerged as a method to enhance efficiency through adaptive circuit selection. However, this approach often leads to alignment degradation, compromising safety by retaining only input-dependent circuits. AAPP addresses these vulnerabilities by adaptively preserving alignment-relevant circuits during inference. Experiments conducted on models such as LLaMA 2-7B, Qwen2.5-14B-Instruct, and Gemma-3-12B-IT demonstrate that AAPP can improve refusal rates by 50% while maintaining matched computational efficiency. This improvement is crucial for ensuring that LLMs can be deployed safely and effectively, highlighting the importance of alignment in AI systems.