arXiv:2511.00833v1 Announce Type: new 
Abstract: Vision Transformers (ViTs) have become a universal backbone for both image recognition and image generation. Yet their Multi-Head Self-Attention (MHSA) layer still performs a quadratic query-key interaction for every token pair, spending the bulk of computation on visually weak or redundant correlations. We introduce Visual-Contrast Attention (VCA), a drop-in replacement for MHSA that injects an explicit notion of discrimination while reducing the theoretical complexity from O(N N C) to O(N n C) with n << N. VCA first distils each head's dense query field into a handful of spatially pooled visual-contrast tokens, then splits them into a learnable positive and negative stream whose differential interaction highlights what truly separates one region from another. The module adds fewer than 0.3M parameters to a DeiT-Tiny backbone, requires no extra FLOPs, and is wholly architecture-agnostic. Empirically, VCA lifts DeiT-Tiny top-1 accuracy on ImageNet-1K from 72.2% to 75.6% (+3.4) and improves three strong hierarchical ViTs by up to 3.1%, while in class-conditional ImageNet generation it lowers FID-50K by 2.1 to 5.2 points across both diffusion (DiT) and flow (SiT) models. Extensive ablations confirm that (i) spatial pooling supplies low-variance global cues, (ii) dual positional embeddings are indispensable for contrastive reasoning, and (iii) combining the two in both stages yields the strongest synergy. VCA therefore offers a simple path towards faster and sharper Vision Transformers. The source code is available at https://github.com/LeapLabTHU/LinearDiff.

تقدم دراسة جديدة Linear Differential Vision Transformer، الذي يعزز كفاءة Vision Transformers من خلال استبدال طبقة Multi-Head Self-Attention التقليدية بـ Visual-Contrast Attention. يركز هذا النهج المبتكر على التباينات البصرية المهمة، مما يقلل من الهدر الحاسوبي على العلاقات الأقل صلة. هذه الخطوة مهمة لأنها تعد بتحسين مهام التعرف على الصور وتوليدها، مما يجعل أنظمة الذكاء الاصطناعي أكثر فعالية.

Un nuevo estudio presenta el Linear Differential Vision Transformer, que mejora la eficiencia de los Vision Transformers al reemplazar la capa tradicional de Multi-Head Self-Attention por la Visual-Contrast Attention. Este enfoque innovador se centra en contrastes visuales significativos, reduciendo el desperdicio computacional en correlaciones menos relevantes. Este avance es significativo ya que promete mejorar tanto las tareas de reconocimiento como de generación de imágenes, haciendo que los sistemas de IA sean más efectivos.

Une nouvelle étude présente le Linear Differential Vision Transformer, qui améliore l'efficacité des Vision Transformers en remplaçant la couche traditionnelle de Multi-Head Self-Attention par la Visual-Contrast Attention. Cette approche innovante se concentre sur des contrastes visuels significatifs, réduisant le gaspillage computationnel sur des corrélations moins pertinentes. Cette avancée est importante car elle promet d'améliorer les tâches de reconnaissance et de génération d'images, rendant les systèmes d'IA plus efficaces.

A new study introduces the Linear Differential Vision Transformer, which enhances the efficiency of Vision Transformers by replacing the traditional Multi-Head Self-Attention layer with Visual-Contrast Attention. This innovative approach focuses on meaningful visual contrasts, reducing computational waste on less relevant correlations. This advancement is significant as it promises to improve both image recognition and generation tasks, making AI systems more effective and efficient.

Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials

One More Thing in AI – Your Shortcut to AI Mastery

Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

WasItAI

The Visualizer

Attentive AI

Tattoo Visualizer

Ready to build your own newsroom?