arXiv:2511.15390v2 Announce Type: replace 
Abstract: Large language models (LLMs) have achieved remarkable performance on a wide range of tasks, hindering real-world deployment due to their massive size. Existing pruning methods (e.g., Wanda) tailored for LLMs rely heavily on manual design pruning algorithms, thereby leading to huge labor costs and requires expert knowledge. Furthermore, we are the first to identify the serious outlier value issue behind dramatic performance degradation under high pruning ratios that are caused by uniform sparsity, raising an additional concern about how to design adaptive pruning sparsity ideal for LLMs. Can LLMs prune by themselves? In this work, we introduce an affirmative answer by proposing a novel pruning method called AutoPrune, which first overcomes expert knowledge limits by leveraging LLMs to design optimal pruning algorithm for themselves automatically without any expert knowledge. Specifically, to mitigate the black-box nature of LLMs, we propose a Graph-driven Chain-of-Thought (GCoT) to optimize prompts, significantly enhancing the reasoning process in learning the pruning algorithm and enabling us to generate pruning algorithms with superior performance and interpretability in the next generation. Finally, grounded in insights of outlier value issue, we introduce Skew-aware Dynamic Sparsity Allocation (SDSA) to overcome the outlier value issue, mitigating performance degradation under high pruning ratios. We conduct extensive experiments on mainstream LLMs benchmarks, demonstrating the superiority of AutoPrune, which consistently excels state-of-the-art competitors.

تم تقديم طريقة جديدة للتقليم تُدعى AutoPrune لنماذج اللغة الكبيرة (LLMs)، والتي تعالج التحديات التي يطرحها حجمها الضخم وطبيعة تقنيات التقليم الحالية التي تتطلب جهدًا كبيرًا. تتيح هذه الطريقة لنماذج اللغة الكبيرة تصميم خوارزميات تقليم مثلى بشكل مستقل دون الحاجة إلى معرفة الخبراء، مما يقلل من التكاليف ويحسن الكفاءة.

Se ha introducido un nuevo método de poda llamado AutoPrune para los Modelos de Lenguaje Grande (LLMs), que aborda los desafíos planteados por su tamaño masivo y la naturaleza laboriosa de las técnicas de poda existentes. Este método permite a los LLMs diseñar de manera autónoma algoritmos de poda óptimos sin requerir conocimientos de expertos, reduciendo así costos y mejorando la eficiencia.

Une nouvelle méthode de taille appelée AutoPrune a été introduite pour les grands modèles de langage (LLMs), abordant les défis posés par leur taille massive et la nature laborieuse des techniques de taille existantes. Cette méthode permet aux LLMs de concevoir de manière autonome des algorithmes de taille optimaux sans nécessiter de connaissances d'expert, réduisant ainsi les coûts et améliorant l'efficacité.

A novel pruning method named AutoPrune has been introduced for Large Language Models (LLMs), addressing the challenges posed by their massive size and the labor-intensive nature of existing pruning techniques. This method allows LLMs to autonomously design optimal pruning algorithms without requiring expert knowledge, thus reducing costs and improving efficiency.