arXiv:2508.07981v4 Announce Type: replace 
Abstract: Visual effects (VFX) are essential visual enhancements fundamental to modern cinematic production. Although video generation models offer cost-efficient solutions for VFX production, current methods are constrained by per-effect LoRA training, which limits generation to single effects. This fundamental limitation impedes applications that require spatially controllable composite effects, i.e., the concurrent generation of multiple effects at designated locations. However, integrating diverse effects into a unified framework faces major challenges: interference from effect variations and spatial uncontrollability during multi-VFX joint training. To tackle these challenges, we propose Omni-Effects, a first unified framework capable of generating prompt-guided effects and spatially controllable composite effects. The core of our framework comprises two key innovations: (1) LoRA-based Mixture of Experts (LoRA-MoE), which employs a group of expert LoRAs, integrating diverse effects within a unified model while effectively mitigating cross-task interference. (2) Spatial-Aware Prompt (SAP) incorporates spatial mask information into the text token, enabling precise spatial control. Furthermore, we introduce an Independent-Information Flow (IIF) module integrated within the SAP, isolating the control signals corresponding to individual effects to prevent any unwanted blending. To facilitate this research, we construct a comprehensive VFX dataset Omni-VFX via a novel data collection pipeline combining image editing and First-Last Frame-to-Video (FLF2V) synthesis, and introduce a dedicated VFX evaluation framework for validating model performance. Extensive experiments demonstrate that Omni-Effects achieves precise spatial control and diverse effect generation, enabling users to specify both the category and location of desired effects.

تم تقديم Omni-Effects كإطار موحد لتوليد المؤثرات البصرية (VFX) الذي يسمح بتوليد مؤثرات مركبة قابلة للتحكم مكانيًا وموجهة عبر التعليمات، مما يعالج القيود المفروضة على طرق إنتاج المؤثرات البصرية الحالية التي تقتصر على توليد مؤثرات فردية. تستخدم هذه الابتكار مجموعة من الخبراء المعتمدة على LoRA لتحسين عملية التوليد.

Omni-Effects se ha presentado como un marco unificado para la generación de efectos visuales (VFX) que permite efectos compuestos guiados por indicaciones y controlables espacialmente, abordando las limitaciones de los métodos de producción de VFX actuales que restringen la generación a efectos individuales. Esta innovación utiliza un Mezcla de Expertos basada en LoRA para mejorar el proceso de generación.

Omni-Effects a été introduit comme un cadre unifié pour la génération d'effets visuels (VFX) permettant des effets composites guidés par des invites et spatialement contrôlables, répondant aux limitations des méthodes de production VFX actuelles qui restreignent la génération à des effets uniques. Cette innovation utilise un mélange d'experts basé sur LoRA pour améliorer le processus de génération.

Omni-Effects has been introduced as a unified framework for generating visual effects (VFX) that allows for prompt-guided and spatially controllable composite effects, addressing limitations in current VFX production methods that restrict generation to single effects. This innovation utilizes LoRA-based Mixture of Experts to enhance the generation process.

Omni-Effects: Unified and Spatially-Controllable Visual Effects Generation

arXiv:2601.07833v2 Announce Type: replace 
Abstract: We present RefVFX, a new framework that transfers complex temporal effects from a reference video onto a target video or image in a feed-forward manner. While existing methods excel at prompt-based or keyframe-conditioned editing, they struggle with dynamic temporal effects such as dynamic lighting changes or character transformations, which are difficult to describe via text or static conditions. Transferring a video effect is challenging, as the model must integrate the new temporal dynamics with the input video's existing motion and appearance. % To address this, we introduce a large-scale dataset of triplets, where each triplet consists of a reference effect video, an input image or video, and a corresponding output video depicting the transferred effect. Creating this data is non-trivial, especially the video-to-video effect triplets, which do not exist naturally. To generate these, we propose a scalable automated pipeline that creates high-quality paired videos designed to preserve the input's motion and structure while transforming it based on some fixed, repeatable effect. We then augment this data with image-to-video effects derived from LoRA adapters and code-based temporal effects generated through programmatic composition. Building on our new dataset, we train our reference-conditioned model using recent text-to-video backbones. Experimental results demonstrate that RefVFX produces visually consistent and temporally coherent edits, generalizes across unseen effect categories, and outperforms prompt-only baselines in both quantitative metrics and human preference. See our website https://tuningfreevisualeffects-maker.github.io/Tuning-free-Visual-Effect-Transfer-across-Videos-Project-Page/

تم تقديم إطار جديد يسمى RefVFX، مما يتيح نقل التأثيرات الزمنية المعقدة من فيديو مرجعي إلى فيديو أو صورة مستهدفة بطريقة مباشرة. تعالج هذه الابتكار التحديات المتعلقة بالتأثيرات الزمنية الديناميكية، مثل تغييرات الإضاءة وتحولات الشخصيات، والتي يصعب التعبير عنها من خلال النص أو الشروط الثابتة.

Se ha presentado un nuevo marco llamado RefVFX que permite la transferencia de efectos temporales complejos de un video de referencia a un video o imagen objetivo de manera directa. Esta innovación aborda los desafíos de los efectos temporales dinámicos, como los cambios de iluminación y las transformaciones de personajes, que son difíciles de describir a través de texto o condiciones estáticas.

Un nouveau cadre nommé RefVFX a été introduit, permettant le transfert d'effets temporels complexes d'une vidéo de référence vers une vidéo ou une image cible de manière directe. Cette innovation répond aux défis des effets temporels dynamiques, tels que les changements d'éclairage et les transformations de personnages, qui sont difficiles à exprimer par le texte ou des conditions statiques.

A new framework named RefVFX has been introduced, enabling the transfer of complex temporal effects from a reference video to a target video or image in a feed-forward manner. This innovation addresses challenges in dynamic temporal effects, such as lighting changes and character transformations, which are difficult to articulate through text or static conditions.

Omni-Effects: Unified and Spatially-Controllable Visual Effects Generation

Was this article worth reading? Share it

Magicley AI

LucidQuery AI

Exaecut

sync. labs

Metaflow AI

Sound Effect Generator

Ready to build your own newsroom?