Schedule

out: Welcome Survey

out: HW1-GP

Course Overview + Gaussian Processes

- Notes: [Linear Regression (PDF)]

Gaussian Processes

- Notes: [Kernels & GPs (PDF)]

Ch. 2 of Rasmussen & Williams GP Textbook

>>> Focus esp. on 2.1-2.5.

- GP Review Material from Stanford CS229;

- David Mackay's 'Humble Gaussian'

Bayesian Neural Nets

- Demo Notebook: [Intro to Autograd and Neural Nets (.ipynb)]

Sec. 1.1 & 1.2 from Neal PhD Thesis

>>> Focus esp. on Fig. 1.2 and its methods

Mackay 1995

>>> Sections 1, 2, 3, & 5

>>> Focus esp. on Sec. 2

Buntine & Weigend 1991

MCMC for BNNs (day 1/2)

- Notes: [MCMC (PDF)]

- Demos: [Random Walk] & [Hamiltonian Monte Carlo]

Sec. 1 - 4.2 of Betancourt 2017

>>> Focus esp. on intuition behind Fig. 8, 9, and 22

For another primer on MCMC, see Sec. 1.3 of Neal PhD Thesis

MCMC for BNNs (day 2/2)

Sec. 1 - 4 of Neal Handbook of HMC

>>> Focus on the algorithm on page 14

>>> Can skip 4.1 altogether

BBVI paper: Ranganath et al. AISTATS 2014

>>> Focus on Sec. 1 and 2 and esp. Alg. 1: could you do this for BNNs? Skim the cool experiments in Sec. 5. Skip Sec. 3&4.

To learn more about VI in general: Review by Blei et al. JASA 2017

Variational for BNNs: Score Function Trick

- Notes: [BBVI for BNNs (PDF)]

"Log Derivative Trick" Blog Post by S. Mohamed 2015

Variational for BNNs: Reparameterization Trick

- Notes: [Reparam. Trick & Bayes-By-Backprop (PDF)]

- Bayes by Backprop: Blundell et al. 2015

>>> Focus on Sec. 1-3 and esp. the Alg. in Sec. 3.2. You can skim/skip Sec. 4 (on contextual bandits). Do read about experiments in Sec. 5, esp. Fig. 5

- "Reparameterisation Tricks" Blog Post by S. Mohamed 2015

An older study of VI methods for BNNs: Graves NIPS 2011

Autoencoders (AEs)

- Demo by Karpathy: [AE for MNIST (web)]

- Notes: [Autoencoders (PDF)]

- Stacked Denoising AEs: Vincent et al. 2010

>>> Focus esp. on Sec. 2.2 and experiments

Overview chapter on AEs: Ch 14 of Goodfellow et al.'s DL Textbook 2016

Cool application in medicine: 'Deep patient' Miotto et al. Sci. Reports 2016

Variational Autoencoders (VAEs) day 1

- Notes: [VAEs (PDF)]

Kingma & Welling ICLR 2014

Tutorial on VAE: Doersch arXiv 2016

Concurrent paper with similar ideas: Rezende et al. ICML 2014

Extension to Graphical Models: Johnson et al. NIPS 2016

due: Final Project Team Signup

due: Discussion Leader Signup

VAEs with discrete random variables

-Presenters: Rohan & Preetish

'Concrete': Maddison et al. ICLR 2017

Concurrent paper with similar idea: Jang et al. ICLR 2017

Semi-supervised VAEs

-Presenters: Alexander & Zhe & Cuong

SS-VAEs: Kingma et al. NIPS 2014

Disentangled representations and VAEs

-Presenters: Hao & Alphonsus & Vladimir

Disentangled VAE: Siddharth et al. NIPS 2017

Dropout as Bayesian Approximation

-Presenters: David & Michael & Andrew

- Dropout as Bayes Approx Paper: Gal & Ghahramani 2016

>>> Focus on understanding Fig. 2 and Fig. 4

- Blog Post by Yarin Gal

>>> Skim and play with interactive demo.

Original Dropout paper: Srivastava et al. JMLR 2014

Kingma et al. 2015

Hron et al. 2017

Modeling Different Kinds of Uncertainty

-Presenters: Stamatios & Ruiyuan & Alexa

Kendall & Gal 2017

Alternatives to BNNs

-Presenters: Victoria & Xiaohui & Emerson

Lakshminarayanan et al. NIPS 2017

Flexible Approximate Posteriors #1

-Presenters: Christian & Daniel & Saber

Normalizing Flows: Rezende & Mohamed ICML 2015

Nonparametric Variational Inference: Gershman et al. ICML 2012

Autoregressive Flows: Kingma et al. NIPS 2016

Multiplicative Flows: Louizos & Welling ICML 2017

Flexible Approximate Posteriors #2

-Presenters: Josh & Holt & Xu

Neural Autoregressive Flows: Huang et al. ICML 2018