arXiv:2512.11532v1 Announce Type: cross 
Abstract: The growing demand for real-time DNN applications on edge devices necessitates faster inference of increasingly complex models. Although many devices include specialized accelerators (e.g., mobile GPUs), dynamic control-flow operators and unsupported kernels often fall back to CPU execution. Existing frameworks handle these fallbacks poorly, leaving CPU cores idle and causing high latency and memory spikes. We introduce Parallax, a framework that accelerates mobile DNN inference without model refactoring or custom operator implementations. Parallax first partitions the computation DAG to expose parallelism, then employs branch-aware memory management with dedicated arenas and buffer reuse to reduce runtime footprint. An adaptive scheduler executes branches according to device memory constraints, meanwhile, fine-grained subgraph control enables heterogeneous inference of dynamic models. By evaluating on five representative DNNs across three different mobile devices, Parallax achieves up to 46% latency reduction, maintains controlled memory overhead (26.5% on average), and delivers up to 30% energy savings compared with state-of-the-art frameworks, offering improvements aligned with the responsiveness demands of real-time mobile inference.

بارالاكس هو إطار جديد مصمم لتحسين كفاءة استدلال الشبكات العصبية العميقة (DNN) على أنظمة الحافة غير المتجانسة، حيث يتناول التحديات التي تطرحها مشغلات التحكم الديناميكي في التدفق التي غالبًا ما تعود إلى التنفيذ على وحدة المعالجة المركزية، مما يؤدي إلى مشكلات في الكمون والذاكرة. يقوم الإطار بتحسين الحساب عن طريق تقسيم تدفق البيانات واستخدام تقنيات متقدمة لإدارة الذاكرة.

Parallax es un nuevo marco diseñado para mejorar la eficiencia de la inferencia de redes neuronales profundas (DNN) en sistemas heterogéneos de borde, abordando los desafíos que plantean los operadores de control de flujo dinámico que a menudo regresan a la ejecución en CPU, lo que provoca problemas de latencia y memoria. El marco optimiza el cálculo al particionar el flujo de datos y emplear técnicas avanzadas de gestión de memoria.

Parallax est un nouveau cadre conçu pour améliorer l'efficacité de l'inférence des réseaux de neurones profonds (DNN) sur des systèmes hétérogènes en périphérie, abordant particulièrement les défis posés par les opérateurs de contrôle de flux dynamique qui reviennent souvent à l'exécution sur CPU, entraînant des problèmes de latence et de mémoire. Le cadre optimise le calcul en partitionnant le flux de données et en utilisant des techniques avancées de gestion de la mémoire.

Parallax is a newly introduced framework designed to enhance the efficiency of deep neural network (DNN) inference on heterogeneous edge systems, particularly addressing the challenges posed by dynamic control-flow operators that often revert to CPU execution, leading to latency and memory issues. The framework optimizes computation by partitioning the data flow and employing advanced memory management techniques.

Parallax: Runtime Parallelization for Operator Fallbacks in Heterogeneous Edge Systems

One More Thing in AI – Your Shortcut to AI Mastery

Parallax: Runtime Parallelization for Operator Fallbacks in Heterogeneous Edge Systems

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

Aqaba.ai

Dynamiq

Dyad

GPTHumanizer

Ready to build your own newsroom?