Skip to content

Latest commit

 

History

History
66 lines (46 loc) · 5.26 KB

README.md

File metadata and controls

66 lines (46 loc) · 5.26 KB
Raw

Improved-DDPM

Improved Denoising Diffusion Probabilistic Models

Abstract

Denoising diffusion probabilistic models (DDPM) are a class of generative models which have recently been shown to produce excellent samples. We show that with a few simple modifications, DDPMs can also achieve competitive log-likelihoods while maintaining high sample quality. Additionally, we find that learning variances of the reverse diffusion process allows sampling with an order of magnitude fewer forward passes with a negligible difference in sample quality, which is important for the practical deployment of these models. We additionally use precision and recall to compare how well DDPMs and GANs cover the target distribution. Finally, we show that the sample quality and likelihood of these models scale smoothly with model capacity and training compute, making them easily scalable. We release our code at this https URL.

Results and Models

Denoising process of Improve-DDPM trained on CIFAR10 and ImageNet-64x64
  
Models Dataset FID Config Download
Improve-DDPM 32x32 Dropout=0.3 CIFAR10 3.8848 config model| log
Improve-DDPM 64x64 ImageNet1k 13.5181 config model| log
Improve-DDPM 64x64 Dropout=0.3 ImageNet1k 13.4094 config model| log

FID comparison with official:

Dataset CIFAR10 ImageNet1k-64x64
Ours 3.8848 13.5181
Official 3.19 19.2

For FID evaluation, we follow the pipeline of BigGAN, where the whole training set is adopted to extract inception statistics, and Pytorch Studio GAN uses 50000 randomly selected samples. Besides, we also use Tero's Inception for feature extraction.

You can download the preprocessed inception state by the following url: CIFAR10 and ImageNet1k-64x64.

You can use following commands to extract those inception states by yourself.

# For CIFAR10
python tools/utils/inception_stat.py --data-cfg configs/_base_/datasets/cifar10_inception_stat.py --pklname cifar10.pkl --no-shuffle --inception-style stylegan --num-samples -1 --subset train

# For ImageNet1k-64x64
python tools/utils/inception_stat.py --data-cfg configs/_base_/datasets/imagenet_64x64_inception_stat.py --pklname imagenet_64x64.pkl --no-shuffle --inception-style stylegan --num-samples -1 --subset train

Citation

Improve-DDPM (arXiv'2021) 
@article{nichol2021improved,
  title={Improved denoising diffusion probabilistic models},
  author={Nichol, Alex and Dhariwal, Prafulla},
  journal={arXiv preprint arXiv:2102.09672},
  year={2021}
}