arXiv:2601.08179v1 Announce Type: cross 
Abstract: A 3D avatar typically has one of six cardinal facial expressions. To simulate realistic emotional variation, we should be able to render a facial transition between two arbitrary expressions. This study presents a new framework for instruction-driven facial expression generation that produces a 3D face and, starting from an image of the face, transforms the facial expression from one designated facial expression to another. The Instruction-driven Facial Expression Decomposer (IFED) module is introduced to facilitate multimodal data learning and capture the correlation between textual descriptions and facial expression features. Subsequently, we propose the Instruction to Facial Expression Transition (I2FET) method, which leverages IFED and a vertex reconstruction loss function to refine the semantic comprehension of latent vectors, thus generating a facial expression sequence according to the given instruction. Lastly, we present the Facial Expression Transition model to generate smooth transitions between facial expressions. Extensive evaluation suggests that the proposed model outperforms state-of-the-art methods on the CK+ and CelebV-HQ datasets. The results show that our framework can generate facial expression trajectories according to text instruction. Considering that text prompts allow us to make diverse descriptions of human emotional states, the repertoire of facial expressions and the transitions between them can be expanded greatly. We expect our framework to find various practical applications More information about our project can be found at https://vohoanganh.github.io/tg3dfet/

تم تقديم إطار عمل جديد لتوليد تعبيرات الوجه ثلاثية الأبعاد المدفوعة بالتعليمات، مما يتيح تحويل تعبيرات وجه شخصية ثلاثية الأبعاد من واحدة إلى أخرى بناءً على التعليمات النصية. يستخدم هذا الإطار مُفكك تعبير الوجه المدفوع بالتعليمات (IFED) لتعلم البيانات متعددة الأنماط وتعزيز العلاقة بين النص وميزات الوجه.

Se ha introducido un nuevo marco para la generación de expresiones faciales 3D impulsada por instrucciones, que permite la transformación de las expresiones faciales de un avatar 3D de una a otra según las instrucciones textuales. Este marco utiliza el Descomponedor de Expresión Facial Impulsado por Instrucción (IFED) para aprender datos multimodales y mejorar la correlación entre el texto y las características faciales.

Un nouveau cadre pour la génération d'expressions faciales 3D pilotée par instruction a été introduit, permettant la transformation des expressions faciales d'un avatar 3D d'une à l'autre en fonction des instructions textuelles. Ce cadre utilise le Décomposeur d'Expression Faciale Piloté par Instruction (IFED) pour apprendre des données multimodales et améliorer la corrélation entre le texte et les caractéristiques faciales.

A new framework for instruction-driven 3D facial expression generation has been introduced, enabling the transformation of a 3D avatar's facial expressions from one to another based on textual instructions. This framework utilizes the Instruction-driven Facial Expression Decomposer (IFED) to learn multimodal data and enhance the correlation between text and facial features.

Instruction-Driven 3D Facial Expression Generation and Transition

Was this article worth reading? Share it

SwapAnything.io

Fakeface

Synthesia

Interactive Avatar

AiReelGenerator.com

ClipCutAi

Ready to build your own newsroom?