arXiv:2409.07414v2 Announce Type: replace 
Abstract: Recent advances in implicit neural representation (INR)-based video coding have demonstrated its potential to compete with both conventional and other learning-based approaches. With INR methods, a neural network is trained to overfit a video sequence, with its parameters compressed to obtain a compact representation of the video content. However, although promising results have been achieved, the best INR-based methods are still out-performed by the latest standard codecs, such as VVC VTM, partially due to the simple model compression techniques employed. In this paper, rather than focusing on representation architectures as in many existing works, we propose a novel INR-based video compression framework, Neural Video Representation Compression (NVRC), targeting compression of the representation. Based on the novel entropy coding and quantization models proposed, NVRC, for the first time, is able to optimize an INR-based video codec in a fully end-to-end manner. To further minimize the additional bitrate overhead introduced by the entropy models, we have also proposed a new model compression framework for coding all the network, quantization and entropy model parameters hierarchically. Our experiments show that NVRC outperforms many conventional and learning-based benchmark codecs, with a 24% average coding gain over VVC VTM (Random Access) on the UVG dataset, measured in PSNR. As far as we are aware, this is the first time an INR-based video codec achieving such performance. The implementation of NVRC will be released.

أدت التطورات الأخيرة في ترميز الفيديو القائم على التمثيل العصبي الضمني (INR) إلى تقديم ضغط تمثيل الفيديو العصبي (NVRC)، الذي يهدف إلى تحسين ضغط الفيديو من خلال تعزيز تقنيات التمثيل. يستخدم هذا الإطار نماذج جديدة للترميز بالانتروبيا والتكميم لتحسين الأساليب الحالية، التي تم تجاوزها بواسطة برامج الترميز التقليدية مثل VVC VTM.

Los avances recientes en la codificación de video basada en representación neuronal implícita (INR) han llevado a la introducción de la Compresión de Representación de Video Neuronal (NVRC), que busca optimizar la compresión de video mejorando las técnicas de representación. Este marco utiliza nuevos modelos de codificación de entropía y cuantificación para mejorar los métodos existentes, que han sido superados por códecs tradicionales como VVC VTM.

Les avancées récentes dans le codage vidéo basé sur la représentation neuronale implicite (INR) ont conduit à l'introduction de la Compression de Représentation Vidéo Neuronale (NVRC), qui vise à optimiser la compression vidéo en améliorant les techniques de représentation. Ce cadre utilise de nouveaux modèles de codage d'entropie et de quantification pour améliorer les méthodes existantes, qui ont été surpassées par des codecs traditionnels comme VVC VTM.

Recent advancements in implicit neural representation (INR)-based video coding have led to the introduction of Neural Video Representation Compression (NVRC), which aims to optimize video compression by enhancing representation techniques. This framework utilizes novel entropy coding and quantization models to improve upon existing methods, which have been outperformed by traditional codecs like VVC VTM.

NVRC: Neural Video Representation Compression

arXiv:2512.04019v1 Announce Type: new 
Abstract: Recent works have demonstrated the viability of utilizing over-fitted implicit neural representations (INRs) as alternatives to autoencoder-based models for neural video compression. Among these INR-based video codecs, Neural Video Representation Compression (NVRC) was the first to adopt a fully end-to-end compression framework that compresses INRs, achieving state-of-the-art performance. Moreover, some recently proposed lightweight INRs have shown comparable performance to their baseline codecs with computational complexity lower than 10kMACs/pixel. In this work, we extend NVRC toward lightweight representations, and propose NVRC-Lite, which incorporates two key changes. Firstly, we integrated multi-scale feature grids into our lightweight neural representation, and the use of higher resolution grids significantly improves the performance of INRs at low complexity. Secondly, we address the issue that existing INRs typically leverage autoregressive models for entropy coding: these are effective but impractical due to their slow coding speed. In this work, we propose an octree-based context model for entropy coding high-dimensional feature grids, which accelerates the entropy coding module of the model. Our experimental results demonstrate that NVRC-Lite outperforms C3, one of the best lightweight INR-based video codecs, with up to 21.03% and 23.06% BD-rate savings when measured in PSNR and MS-SSIM, respectively, while achieving 8.4x encoding and 2.5x decoding speedup. The implementation of NVRC-Lite will be made available.

أدت التطورات الأخيرة في ضغط الفيديو العصبي إلى تطوير NVRC-Lite، وهو امتداد لضغط تمثيل الفيديو العصبي (NVRC). يتضمن هذا الإطار الجديد شبكات ميزات متعددة المقاييس وشبكات عالية الدقة، مما يحسن الأداء بشكل كبير مع الحفاظ على تعقيد حسابي منخفض.

Los avances recientes en la compresión de video neuronal han llevado al desarrollo de NVRC-Lite, una extensión de la Compresión de Representación de Video Neuronal (NVRC). Este nuevo marco integra cuadrículas de características a múltiples escalas y cuadrículas de mayor resolución, mejorando significativamente el rendimiento mientras mantiene una baja complejidad computacional.

Des avancées récentes dans la compression vidéo neuronale ont conduit au développement de NVRC-Lite, une extension de la Compression de Représentation Vidéo Neuronale (NVRC). Ce nouveau cadre intègre des grilles de caractéristiques multi-échelles et des grilles de haute résolution, améliorant considérablement les performances tout en maintenant une faible complexité computationnelle.

Recent advancements in neural video compression have led to the development of NVRC-Lite, an extension of Neural Video Representation Compression (NVRC). This new framework integrates multi-scale feature grids and higher resolution grids, significantly enhancing performance while maintaining low computational complexity.

NVRC: Neural Video Representation Compression

Was this article worth reading? Share it

LucidQuery AI

Videotok

VideoDigest