Schedule

Schedule might change slightly from that listed here. Please check back regularly.

For any class day with assigned readings, you should submit critical comments

- Comments are due by 8 a.m. on the day we'll discuss the readings
- Post to the course-specific Canvas discussion forum
- Detailed instructions: assignments.html#comments-on-readings
- If there are multiple 'required' readings that day, you only need comments on the first one listed.

Part 1: Bayesian Neural Networks

How do we model functions?

date	assignments	topic	required	optional
Tue 09/04	out: Welcome Survey out: HW1-GP	Course Overview + Gaussian Processes • Notes: [Linear Regr. (PDF)]
Thu 09/06		Gaussian Processes • Notes: [Kernels & GPs (PDF)]	Ch. 2 of Rasmusen & Williams GP Textbook - Focus esp. on 2.1-2.5.	GP Review Material from Stanford CS229; David Mackay's 'Humble Gaussian' has colorful algebra
Tue 09/11	due: HW1-GP out: HW2-BNN+HMC	Bayesian Neural Nets • Code: [Intro to autograd and NN training (ipynb)]	Sec. 1.1 & 1.2 from Neal PhD Thesis - Focus esp. on Fig. 1.2 and its methods Sec. 1, 2, 3, and 5 from Mackay 1995 - Focus esp. on Sec. 2	Buntine & Weigend 1991
Thu 09/13		MCMC Training for BNNs • Notes: [MCMC (PDF)] • Demos: [Random Walk] & [Hamiltonian Monte Carlo]	Sec. 3 & 4 of Neal Handbook of HMC - esp. the algorithm on page 14 Sec. 1-5 of Betancourt 2017 -- esp. intuition in Fig. 22	For a refresh on MCMC, see Sec. 1.3 of Neal PhD Thesis
Tue 09/18	out: HW3-BNN+VI HW2 DUE DATE EXTENDED TO THURS	MCMC with HMC (day 2/2) • Code: [NN prior, likelihood, and posterior (ipynb)]	- BBVI paper: Ranganath et al. AISTATS 2014 >>> Focus on Sec. 1 and 2 and esp. Alg. 1: could you do this for BNNs? Skim the cool experiments in Sec. 5. Feel free to skip Sec. 3&4.	To learn more about VI in general: Review by Blei et al. JASA 2017
Thu 09/20	due: HW2-BNN+HMC	Variational Training 1 for BNNs: Black-box VI • Notes: [BBVI for BNNs (PDF)]	"Log Derivative Trick" Blog Post by S. Mohamed 2015
Tue 09/25		Variational Training 2 for BNNs: Reparameterization Trick • Notes: [Reparam. Trick & Bayes-By-Backprop (PDF)]	- Bayes by Backprop: Blundell et al. 2015 >>> Focus on Sec. 1-3 and esp. the Alg. in Sec. 3.2. You can skim/skip Sec. 4 (on contextual bandits). Do read about experiments in Sec. 5, esp. Fig. 5 - "Reparameterisation Tricks" Blog Post by S. Mohamed 2015	An older study of VI methods for BNNs: Graves NIPS 2011
Thu 09/27	out:Initial Pitch Checkpoint1	Bayesian Nonparametric Machine Learning for Discrete Random Structures, Invited Talk by Diana Cai. Location: Halligan 102. Tufts CS Rising Star Research Seminar.	• Diana's short NIPS Networks 2015 workshop paper 'Edge-Exchangable Graphs and Sparsity' >>> Come with 1 good question to ask during the seminar

Part 2: Autoencoders, Deep Generative Models, and VAEs

How do we model structured data?

date	assignments	topic	required	optional
Tue 10/02	due: HW3-BNN+VI out: HW4-VAE	Autoencoders (AEs) • Demo by Karpathy: [AE for MNIST (web)] • Notes: [Autoencoders (PDF)]	- Stacked Denoising AEs: Vincent et al. 2010 >>> Focus esp. on Sec. 2.2 and experiments	Overview chapter on AEs: Ch 14 of Goodfellow et al.'s DL Textbook 2016 Cool application in medicine: 'Deep patient' Miotto et al. Sci. Reports 2016
Thu 10/04		Variational Autoencoders (VAEs) • Notes: [VAEs (PDF)]	Kingma & Welling ICLR 2014	Tutorial on VAE: Doersch arXiv 2016 Concurrent paper with similar ideas: Rezende et al. ICML 2014 Extension to Graphical Models: Johnson et al. NIPS 2016
Tue 10/09		NO CLASS (Monday-on-Tuesday schedule per Tufts Academic Calendar )

Part 3: Research Frontiers

date	assignments	topic	required	optional
Thu 10/11	due: Final Project Team Signup on Canvas	Priors for BNN with Special Guest Dr. Soumya Ghosh • Slides: [BNN with Horseshoe Prior & Structured VI (PDF)]	Ghosh et al. ICML 2018
Tue 10/16	due:Initial Pitch due: HW4-VAE	PROJECT CHECKPOINT 1: In-Class Project Pitches + Feedback
Thu 10/18	out: Checkpoint2	VAEs with discrete random variables -Presenters: Yirong Tang and Edwin Jain	'Concrete': Maddison et al. ICLR 2017	Concurrent paper with similar idea: Jang et al. ICLR 2017
Tue 10/23	due: Checkpoint1	Semi-supervised VAEs -Presenters: Ramtin Hossein and Gyan Tatiya	SS-VAEs: Kingma et al. NIPS 2014
Thu 10/25		Disentangled representations and VAEs -Presenters: Yushi Liu & Linfeng Liu	Disentangled VAE: Siddharth et al. NIPS 2017
Tue 10/30		Dropout as Bayesian Approximation -Presenters: Jamie Heller & Minh D. Nguyen	- Dropout as Bayes Approx Paper: Gal & Ghahramani 2016 >>> Focus on understanding Fig. 2 and Fig. 4 - Blog Post by Yarin Gal >>> Skim and play with interactive demo.	Original Dropout paper: Srivastava et al. JMLR 2014 Kingma et al. 2015 Hron et al. 2017
Thu 11/01		Modeling Different Kinds of Uncertainty -Presenters: Eric K. Wyss & Victor Oludare	Kendall & Gal 2017
Tue 11/06	due: Checkpoint2 out: Checkpoint3	Alternatives to BNNs -Presenters: Jeremy Shih & Duc D. Nguyen	Lakshminarayanan et al. NIPS 2017
Thu 11/08		Improved Variational Objectives -Presenters: Boyang Lyu & Daniel Dinjian	Importance Weighted VAEs: Burda et al. ICLR 2016	Filtering Variational: Maddison et al. NIPS 2017 Are Tighter Bounds Better?: Rainforth BDL 2017
Tue 11/13		Flexible Approximate Posteriors #1 -Presenters: Jong Seo Yoon & Manh Duc Nguyen	Normalizing Flows: Rezende & Mohamed ICML 2015	Nonparametric Variational Inference: Gershman et al. ICML 2012 Autoregressive Flows: Kingma et al. NIPS 2016 Multiplicative Flows: Louizos & Welling ICML 2017
Thu 11/15		PROJECT CHECKPOINT 3 in-class work time
Tue 11/20	due: Checkpoint3 out: Final Report out: Final Presentation	Flexible Approximate Posteriors #2 -Presenters: Julie Jiang & Phong Hoang	Neural Autoregressive Flows: Huang et al. ICML 2018
Thu 11/22		THANKSGIVING HOLIDAY (no class)
Tue 11/27		Course Feedback (first 15 min.) PROJECT WORK TIME
Thu 11/29		CS Dept. 'Rising Stars' Research Seminar: Peter Schulam

Part 4: Final Project Deliverables

date	assignments	topic	required	optional
Tue 12/04	due: Final Presentation	FINAL PROJECT PRESENTATIONS (Part 1/2)
Thu 12/06	due: Final Presentation	FINAL PROJECT PRESENTATIONS (Part 2/2)
Tue 12/18	due: Final Report	FINAL PROJECT REPORT DUE