Enhancing Diffusion Model Guidance through Calibration and Regularization

The paper titled 'Enhancing Diffusion Model Guidance through Calibration and Regularization' presents significant advancements in classifier-guided diffusion models, which are essential for conditional image generation. The authors identify a critical problem: overconfident predictions during early denoising steps that lead to ineffective guidance gradients. To combat this, they propose a differentiable calibration objective based on the Smooth Expected Calibration Error, which enhances classifier calibration with minimal fine-tuning. Additionally, they introduce innovative sampling guidance methods that do not require retraining existing classifiers. These methods include tilted sampling with batch-level reweighting and adaptive entropy-regularized sampling, which help maintain diversity in generated images. The experiments conducted on the ImageNet 128x128 dataset demonstrate that their divergence-regularized guidance achieves an impressive FID of 2.13 using a ResNet-101 classifier, …