Skip to content

This resposity maintains a series of papers on deep generative model.

Notifications You must be signed in to change notification settings

zju-pi/Deep-Generative-Model-Paper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

45 Commits
 
 

Repository files navigation

Contents

Resources

Score-based models

  • Estimation of Non-Normalized Statistical Models by Score Matching [JMLR 2005]

    • Aapo Hyvarinen.
  • Extracting and Composing Robust Features with Denoising Autoencoders [ICML 2008]

    • Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre-Antoine Manzagol.
  • A Connection Between Score Matching and Denoising Autoencoders [Neural Computation 2011]

    • Pascal Vincent.
  • Sliced Score Matching: A Scalable Approach to Density and Score Estimation [UAI 2019][PyTorch]

    • Yang Song, Sahaj Garg, Jiaxin Shi, Stefano Ermon.
  • Generative Modeling by Estimating Gradients of the Data Distribution [NIPS 2019][PyTorch]

    • Yang Song, Stefano Ermon.
  • Improved Techniques for Training Score-Based Generative Models [NIPS 2020][PyTorch]

    • Yang Song, Stefano Ermon.
  • Efficient Learning of Generative Models via Finite-Difference Score Matching [NIPS 2020][PyTorch]

    • Tianyu Pang, Kun Xu, Chongxuan Li, Yang Song, Stefano Ermon, Jun Zhu.
  • Adversarial score matching and improved sampling for image generation [ICLR 2021][PyTorch]

    • Alexia Jolicoeur-Martineau, Rémi Piché-Taillefer, Ioannis Mitliagkas, Remi Tachet des Combes.
  • Score-Based Generative Modeling through Stochastic Differential Equations [ICLR 2021][TensorFlow][PyTorch]

    • Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole.

Sampling

  • An Introduction to MCMC for Machine Learning [ML 2003]

    • Christophe Andrieu, Nando de Freitas, Arnaud Doucet, Michael I. Jordan.
  • Introduction to Markov Chain Monte Carlo [Handbook of Markov Chain Monte Carlo 2010]

    • Charles J. Geyer.
  • MCMC Using Hamiltonian Dynamics [Handbook of Markov Chain Monte Carlo 2010]

    • Radford M. Neal.
  • Bayesian Learning via Stochastic Gradient Langevin Dynamics [ICML 2011]

    • Max Welling, Yee Whye Teh. (Test-of-time Award of ICML 2021)
  • Bayesian Posterior Sampling via Stochastic Gradient Fisher Scoring [ICML 2011]

    • Sungjin Ahn, Anoop Korattikara, Max Welling. (Best Paper Award)

Diffusion models

  • Deep Unsupervised Learning using Nonequilibrium Thermodynamics [ICML 2015]

    • Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, Surya Ganguli.
  • Denoising Diffusion Probabilistic Models [NIPS 2020][TensorFlow]

    • Jonathan Ho, Ajay Jain, Pieter Abbeel.
  • Diffusion Models Beat GANs on Image Synthesis [NIPS 2021]

    • Prafulla Dhariwal, Alex Nichol.

Other models

  • Auto-Encoding Variational Bayes [ICLR 2014]

    • Diederik P Kingma, Max Welling.
  • Generative Adversarial Nets [NIPS 2014]

    • Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio.

Blogs

https://www.inference.vc/denoising-as-unsupervised-learning/

https://openai.com/blog/generative-models/

My Reading list Before 2019

  • ICML 14 -- Danilo J. Rezende et al.

    • (DLGM) Stochastic Backpropagation and Approximate Inference in Deep Generative Models.
    • [Paper]
  • NIPS 14 -- Diederik P. Kingma et al.

    • (SSL-VAE) Semi-Supervised Learning with Deep Generative Models.
    • [Paper]
  • NIPS 15 -- Kihyuk Sohn et al.

    • (CVAE) Learning Structured Output Representation using Deep Conditional Generative Models.
    • [Paper & Supplement]
  • ICLR 17 oral -- Martin Arjovsky et al.

    • (WGAN-1) Towards Principled Methods for Training Generative Adversarial Networks.
    • [Paper]
  • ICML 17 -- Martin Arjovsky et al.

    • (WGAN-2) Wasserstein GAN.
    • [Paper]
  • ICLR 16 -- Jost Tobias Springenberg.

    • (CatGAN) Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks.
    • [Paper]
  • NIPS 16 -- Sebastian Nowozin et al.

    • f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization.
    • [Paper]
  • NIPS 16 -- Xi Chen et al.

    • InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets.
    • [Paper]
  • ICLR 18 -- Zhiting Hu et al.

    • On Unifying Deep Generative Models.
    • [Paper]
  • SIGIR 17 Best Paper-- Jun Wang et al.

    • (IRGAN)IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models.
    • [Paper]
  • WWW 18 -- Dawen Liang et al.

    • Variational Autoencoders for Collaborative Filtering.
    • [Paper]
  • IJCAI 17 -- Zhuxi Jiang et al.

    • (VaDE) Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering.
    • [Paper]
  • AAAI 18 -- Hongwei Wang et al.

    • (GraphGAN) GraphGAN: Graph Representation Learning with Generative Adversarial Nets.
    • [Paper]
  • AAAI 19 -- Sudipto Mukherjee et al.

    • (ClusterGAN) ClusterGAN: Latent Space Clustering in Generative Adversarial Networks.
    • [Paper]
  • NIPS 18 -- Yucen Luo et al.

    • Semi-crowdsourced Clustering with Deep Generative Models.
    • [Paper]
  • JMLR 13 -- Matthew D. Hoffman et al.

    • (SVI) Stochastic Variational Inference.
    • [Paper]
  • ICLR 18 -- Ilya Tolstikhin et al.

    • Wasserstein Auto-Encoders.
    • [Paper]

Tutorial & Review & Lecture Notes

  • JASA 17 -- David M.Blei et al.

  • NIPS 16 Tutorial -- David Blei et al.

  • NIPS 16 Tutorial -- Ian Goodfellow et al.

  • Arxiv 16 -- Carl Doersch.

    • Tutorial on Variational Autoencoders.
    • [Paper]
  • IJCAI 18 Tutorial -- Stefano Ermon et al.

    • Tutorial on Deep Generative Models
    • [Slides]
  • UAI 17 Tutorial -- Shakir Mohamed and Danilo Rezende.

Useful Resources

  • Hung-yi Lee

    • Machine Learning and having it deep and structured (2018, Spring, National Taiwan University)
    • [Notes & Video]
  • Stefano Ermon, Aditya Grover

    • CS 236: Deep Generative Models (2018, Fall, Stanford University)
    • [Notes]
  • PRML

    • Pattern Recognition and Machine Learning. Christopher Bishop.
    • [Book & Support]

About

This resposity maintains a series of papers on deep generative model.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published