Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspective

The recent study on continual unlearning in text-to-image diffusion models highlights a significant issue: popular unlearning methods experience rapid utility collapse when faced with sequential requests. This collapse is traced back to cumulative parameter drift from the model's pre-training weights, which leads to a loss of retained knowledge and degraded image generation. To combat this, the researchers advocate for the use of regularization techniques, which are crucial for mitigating drift and maintaining model performance. They introduce a suite of add-on regularizers and emphasize the necessity of semantic awareness to preserve concepts relevant to the unlearning target. A novel gradient-projection method is proposed, which constrains parameter drift orthogonal to their subspace, significantly enhancing continual unlearning performance. This research not only addresses a pressing challenge in AI model training but also sets the stage for future advancements in machine unlearning…