arXiv:2511.17693v1 Announce Type: cross 
Abstract: Transformer-based models have dramatically increased their size and parameter count to tackle increasingly complex tasks. At the same time, there is a growing demand for low-latency inference on resource-constrained devices that achieves high performance. In particular, stream data inference is typically performed over a sliding temporal window, leading to highly redundant computations. The recent Continual Transformers have addressed this issue, but they can only be effectively used in shallow models, which limits their scope and generalization power. In this paper, we propose the Deep Continual Transformer (DeepCoT), a redundancy-free encoder-only model that can be applied over existing deep encoder architectures with minimal changes. In our experiments over audio, video, and text streams, we show that DeepCoTs retain comparative performance to their non-continual baselines while offering a linear computational cost for all Transformer layers, which reduces up to two orders of magnitude in the running time compared to previous efficient models.

يمثل تقديم DeepCoT، أو المحولات المستمرة العميقة، تقدمًا كبيرًا في الاستدلال في الوقت الحقيقي على تدفقات البيانات، حيث يتناول التحديات المتعلقة بالازدواجية في الحسابات المرتبطة بالنوافذ الزمنية المتحركة. يحافظ هذا النموذج على أداء مقارن مع المحولات التقليدية بينما يقدم تكاليف حسابية خطية عبر أنواع بيانات مختلفة، بما في ذلك الصوت والفيديو والنص.

La introducción de DeepCoT, o Deep Continual Transformers, representa un avance significativo en la inferencia en tiempo real sobre flujos de datos, abordando los desafíos de redundancia en los cálculos asociados con las ventanas temporales deslizantes. Este modelo mantiene un rendimiento comparable al de los transformadores tradicionales mientras ofrece costos computacionales lineales en varios tipos de datos, incluidos audio, video y texto.

L'introduction de DeepCoT, ou Deep Continual Transformers, représente une avancée significative dans l'inférence en temps réel sur les flux de données, abordant les défis de redondance dans les calculs associés aux fenêtres temporelles glissantes. Ce modèle maintient des performances comparables à celles des transformateurs traditionnels tout en offrant des coûts computationnels linéaires sur divers types de données, y compris l'audio, la vidéo et le texte.

The introduction of DeepCoT, or Deep Continual Transformers, presents a significant advancement in real-time inference on data streams, addressing the challenges of redundancy in computations associated with sliding temporal windows. This model maintains performance comparable to traditional transformers while offering linear computational costs across various data types, including audio, video, and text.

DeepCoT: Deep Continual Transformers for Real-Time Inference on Data Streams

Was this article worth reading? Share it

Synthesia

Agentcloud

BlitzToksAI