arXiv:2105.06421v4 Announce Type: replace-cross 
Abstract: Facial emotion recognition (FER) is a fine-grained problem where the value of transfer learning is often assumed. We first quantify this assumption and show that, on AffectNet, training from random initialization with sufficiently strong augmentation consistently matches or surpasses fine-tuning from ImageNet. Motivated by this result, we propose Hybrid Multi-Task Learning (HMTL) for FER in the wild. HMTL augments supervised learning (SL) with self-supervised learning (SSL) objectives during training, while keeping the inference-time model unchanged. We instantiate HMTL with two tailored pretext tasks, puzzling and inpainting with a perceptual loss, that encourage part-aware and expression-relevant features. On AffectNet, both HMTL variants achieve state-of-the-art accuracy in the eight-emotion setting without any additional pretraining data, and they provide larger gains under low-data regimes. Compared with conventional SSL pretraining, HMTL yields stronger downstream performance. Beyond FER, the same strategy improves fine-grained facial analysis tasks, including head pose estimation and gender recognition. These results suggest that aligned SSL auxiliaries are an effective and simple way to strengthen supervised fine-grained facial representation without adding extra computation cost during inference time.

يتناول المقال التقدم في التعرف على مشاعر الوجه (FER) من خلال استخدام التعلم المتعدد المهام الهجين (HMTL). يبرز أن التدريب من نقطة البداية العشوائية مع تعزيز قوي يمكن أن يساوي أو يتجاوز النتائج الناتجة عن تحسين النماذج على ImageNet، خاصة على مجموعة بيانات AffectNet. يجمع HMTL بين التعلم تحت الإشراف مع أهداف التعلم الذاتي، مستخدمًا مهام مثل الألغاز وإعادة البناء لتحسين استخراج الميزات. تحقق الطرق المقترحة دقة رائدة في التعرف على ثماني مشاعر دون الحاجة إلى بيانات تدريب إضافية، مما يفيد بشكل خاص السيناريوهات ذات البيانات القليلة.

El artículo aborda los avances en el reconocimiento de emociones faciales (FER) mediante el uso de Aprendizaje Híbrido Multitarea (HMTL). Se destaca que el entrenamiento desde una inicialización aleatoria con una fuerte augmentación puede igualar o superar los resultados del ajuste fino en ImageNet, especialmente en el conjunto de datos AffectNet. HMTL combina el aprendizaje supervisado con objetivos de aprendizaje auto-supervisado, utilizando tareas como rompecabezas y restauración para mejorar la extracción de características. Los métodos propuestos logran una precisión de vanguardia en el r…

L'article traite des avancées dans la reconnaissance des émotions faciales (FER) grâce à l'utilisation de l'apprentissage hybride multi-tâches (HMTL). Il souligne que l'entraînement à partir d'une initialisation aléatoire avec une forte augmentation peut égaler ou dépasser les résultats de l'affinage sur ImageNet, en particulier sur le jeu de données AffectNet. HMTL combine l'apprentissage supervisé avec des objectifs d'apprentissage auto-supervisé, utilisant des tâches telles que le puzzle et le remplissage pour améliorer l'extraction des caractéristiques. Les méthodes proposées atteignent un…

The article discusses advancements in facial emotion recognition (FER) through the use of Hybrid Multi-Task Learning (HMTL). It highlights that training from random initialization with strong augmentation can match or exceed results from fine-tuning on ImageNet, particularly on the AffectNet dataset. HMTL combines supervised learning with self-supervised learning objectives, utilizing tasks like puzzling and inpainting to enhance feature extraction. The proposed methods achieve state-of-the-art accuracy in recognizing eight emotions without additional pretraining data, especially benefiting lo…

Using Self-Supervised Auxiliary Tasks to Improve Fine-Grained Facial Representation

arXiv:2512.06377v1 Announce Type: new 
Abstract: Current FER (Facial Expression Recognition) dataset is mostly labeled by emotion categories, such as happy, angry, sad, fear, disgust, surprise, and neutral which are limited in expressiveness. However, future affective computing requires more comprehensive and precise emotion metrics which could be measured by VAD(Valence-Arousal-Dominance) multidimension parameters. To address this, AffectNet has tried to add VA (Valence and Arousal) information, but still lacks D(Dominance). Thus, the research introduces VAD annotation on FER2013 dataset, takes the initiative to label D(Dominance) dimension. Then, to further improve network capacity, it enforces orthogonalized convolution on it, which extracts more diverse and expressive features and will finally increase the prediction accuracy. Experiment results show that D dimension could be measured but is difficult to obtain compared with V and A dimension no matter in manual annotation or regression network prediction. Secondly, the ablation test by introducing orthogonal convolution verifies that better VAD prediction could be obtained in the configuration of orthogonal convolution. Therefore, the research provides an initiative labelling for D dimension on FER dataset, and proposes a better prediction network for VAD prediction through orthogonal convolution. The newly built VAD annotated FER2013 dataset could act as a benchmark to measure VAD multidimensional emotions, while the orthogonalized regression network based on ResNet could act as the facial expression recognition baseline for VAD emotion prediction. The newly labeled dataset and implementation code is publicly available on https://github.com/YeeHoran/VAD-Net .

تقدم دراسة جديدة VAD-Net، وهو نهج متعدد الأبعاد للتعرف على تعبيرات الوجه (FER) الذي يعزز مجموعة بيانات FER2013 الحالية من خلال دمج البعد D (الهيمنة) جنبًا إلى جنب مع مقاييس القيمة والاهتمام. يهدف هذا التقدم إلى تحسين دقة التعرف على المشاعر في أنظمة التعليم الذكي، مع معالجة قيود طرق تصنيف المشاعر الحالية.

Un nuevo estudio presenta VAD-Net, un enfoque multidimensional para el reconocimiento de expresiones faciales (FER) que mejora el conjunto de datos FER2013 existente al incorporar la dimensión D (Dominancia) junto con las métricas de Valencia y Arousal. Este avance tiene como objetivo mejorar la precisión del reconocimiento emocional en sistemas educativos inteligentes, abordando las limitaciones de los métodos actuales de categorización emocional.

Une nouvelle étude présente VAD-Net, une approche multidimensionnelle de la reconnaissance des expressions faciales (FER) qui améliore le jeu de données FER2013 existant en intégrant la dimension D (Dominance) aux métriques de Valence et d'Arousal. Cette avancée vise à améliorer la précision de la reconnaissance des émotions dans les systèmes éducatifs intelligents, en répondant aux limites des méthodes de catégorisation des émotions actuelles.

A new study introduces VAD-Net, a multidimensional approach to facial expression recognition (FER) that enhances the existing FER2013 dataset by incorporating the D (Dominance) dimension alongside Valence and Arousal metrics. This advancement aims to improve the precision of emotion recognition in intelligent education systems, addressing the limitations of current emotion categorization methods.

Using Self-Supervised Auxiliary Tasks to Improve Fine-Grained Facial Representation

Was this article worth reading? Share it

Humanize AI

Fakeface

X Headshot