arXiv:2601.08539v1 Announce Type: cross 
Abstract: The rapid growth of AI has fueled the expansion of accelerator- or GPU-based data centers. However, the rising operational energy consumption has emerged as a critical bottleneck and a major sustainability concern. Dynamic Voltage and Frequency Scaling (DVFS) is a well-known technique used to reduce energy consumption, and thus improve energy-efficiency, since it requires little effort and works with existing hardware. Reducing the energy consumption of training and inference of Large Language Models (LLMs) through DVFS or power capping is feasible: related work has shown energy savings can be significant, but at the cost of significant slowdowns. In this work, we focus on reducing waste in LLM operations: i.e., reducing energy consumption without losing performance. We propose a fine-grained, kernel-level, DVFS approach that explores new frequency configurations, and prove these save more energy than previous, pass- or iteration-level solutions. For example, for a GPT-3 training run, a pass-level approach could reduce energy consumption by 2% (without losing performance), while our kernel-level approach saves as much as 14.6% (with a 0.6% slowdown). We further investigate the effect of data and tensor parallelism, and show our discovered clock frequencies translate well for both. We conclude that kernel-level DVFS is a suitable technique to reduce waste in LLM operations, providing significant energy savings with negligible slow-down.

اقترحت دراسة جديدة نهجًا لتقليل استهلاك الطاقة من خلال ضبط الجهد والتردد على مستوى النواة (DVFS) في عمليات نماذج اللغة الكبيرة (LLMs) مثل GPT-3. تهدف هذه الطريقة إلى تقليل هدر الحوسبة دون فقدان الأداء، مما يعالج المخاوف الحرجة المتعلقة بالاستدامة المرتبطة بزيادة الطلب على الطاقة في مراكز البيانات المدعومة بالذكاء الاصطناعي.

Un nuevo estudio ha propuesto un enfoque de escalado dinámico de voltaje y frecuencia (DVFS) a nivel de núcleo, destinado a reducir el consumo de energía en las operaciones de los Modelos de Lenguaje Grande (LLMs) como GPT-3. Este método busca minimizar el desperdicio de computación sin sacrificar el rendimiento, abordando las preocupaciones críticas de sostenibilidad asociadas con el aumento de la demanda energética de los centros de datos impulsados por IA.

Une nouvelle étude a proposé une approche de mise à l'échelle dynamique de la tension et de la fréquence (DVFS) au niveau du noyau, visant à réduire la consommation d'énergie dans les opérations des modèles de langage de grande taille (LLMs) comme GPT-3. Cette méthode cherche à minimiser le gaspillage informatique sans sacrifier les performances, répondant ainsi aux préoccupations critiques de durabilité liées à l'augmentation des besoins énergétiques des centres de données alimentés par l'IA.

A new study has proposed a fine-grained, kernel-level Dynamic Voltage and Frequency Scaling (DVFS) approach aimed at reducing energy consumption in the operations of Large Language Models (LLMs) like GPT-3. This method seeks to minimize compute waste without sacrificing performance, addressing the critical sustainability concerns associated with the rising energy demands of AI-driven data centers.

Reducing Compute Waste in LLMs through Kernel-Level DVFS

One More Thing in AI – Your Shortcut to AI Mastery

Reducing Compute Waste in LLMs through Kernel-Level DVFS

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

Aqaba.ai

Langfuse

GPTHumanizer

GPT Bypass

Ready to build your own newsroom?