Guiding a Diffusion Model with a Bad Version of Itself

The paper introduces "autoguidance," a method that improves the quality of images generated by diffusion models by using a less-trained version of the model as a guide, thus maintaining diversity in the outputs and achieving record FID scores on ImageNet generation tasks.

Abstract

The primary axes of interest in image-generating diffusion models are image quality, the amount of variation in the results, and how well the results align with a given condition, e.g., a class label or a text prompt. The popular classifier-free guidance approach uses an unconditional model to guide a conditional model, leading to simultaneously better prompt alignment and higher-quality images at the cost of reduced variation. These effects seem inherently entangled, and thus hard to control. We make the surprising observation that it is possible to obtain disentangled control over image quality without compromising the amount of variation by guiding generation using a smaller, less-trained version of the model itself rather than an unconditional model. This leads to significant improvements in ImageNet generation, setting record FIDs of 1.01 for 64x64 and 1.25 for 512x512, using publicly available networks. Furthermore, the method is also applicable to unconditional diffusion models, drastically improving their quality.

Method

The authors used a novel methodology called "autoguidance" to improve the quality of images generated by diffusion models. This methodology involves using a smaller, less-trained version of the main model as a guiding model to control the generation process. By doing so, they were able to maintain the diversity of the generated images while significantly improving their quality, as demonstrated by the record-setting FID scores achieved on ImageNet generation tasks.

Main Finding

The authors discovered that by using a smaller, less-trained version of the main diffusion model as a guiding model, they could achieve disentangled control over image quality without compromising the amount of variation in the generated images. This method, termed "autoguidance," led to significant improvements in image quality, setting new records for FID scores on ImageNet generation tasks at both 64x64 and 512x512 resolutions. They also found that this approach could be applied to both conditional and unconditional diffusion models, drastically improving their quality.

Conclusion

The conclusion of the paper is that the authors have successfully introduced a new method called "autoguidance" which can be used to improve the quality of images generated by diffusion models without reducing the diversity of the outputs. This method uses a less-trained version of the main model as a guide, and it has been shown to be effective through various tests and experiments. The authors have also made their implementation and pre-trained models publicly available, and they have discussed the broader societal implications of their work. They suggest that future work could focus on formally proving the conditions for autoguidance and establishing rules for selecting the best guiding model.

Keywords

Image-generating diffusion models, classifier-free guidance (CFG), autoguidance, ImageNet generation, FID scores, denoising diffusion, score function, conditional and unconditional models, model capacity, training time, hyperparameter sensitivity, societal impact, generative modeling.