Resources

Jump to: Software • Conferences & Workshops • Related Courses • Prereq Catchup • Deep Learning Self-study Resources

Software

For this course, we strongly recommend using a custom environment of Python packages all installed and maintained via the free 'conda' environment manager from Anaconda, Inc.: https://conda.io/docs/user-guide/getting-started.html

For detailed instructions, see the [Python Setup Instructions page]

High Performance Computing

For your final project, you can use the Tufts High performance computing cluster

• See the course-specific [Tufts HPC Setup] page.

If you are looking for basic workshops to get help on things like using Linux or the Tufts High-performance computing cluster, checkout the programs in the Tufts Data Lab: https://sites.tufts.edu/datalab/workshops/

Conferences and Workshops

Bayesian Deep Learning workshop at NeurIPS

Good place to browse for potential project ideas:

• 2019 workshop: http://bayesiandeeplearning.org/2019
• 2018 workshop: http://bayesiandeeplearning.org/2018
• 2017 workshop: http://bayesiandeeplearning.org/2017
• 2016 workshop: http://bayesiandeeplearning.org/2016

Previous Versions of this course

This course has been offered twice before at Tufts:

• Fall 2019, taught by Prof. Mike Hughes
• Fall 2018, taught by Prof. Mike Hughes

Related Courses

Harvard COMPSCI 282R: Topics in Machine Learning - Deep Bayesian Models

Taught by Prof. Finale Doshi-Velez, Fall 2018

Stanford CS 236: Deep Generative Models

Taught by Prof. Stefano Ermon, Fall 2019

https://deepgenerativemodels.github.io/

Summer School on Deep Learning and Bayesian Methods

August 27th – September 1st 2018, Moscow, Russia

• Website: http://deepbayes.ru/
• [Videos on Youtube] | [Slides on GDrive] | [Practicals on Github]

Prereq Catchup Resources

Here are some useful resources to help you catch up if you are missing some of the pre-requisite knowledge. Please contribute new resources by starting a topic on the class discussion forum.

To achieve this objective, we expect students to be familiar with:

Probability

• Key concepts:
• • Gaussian pdf (univariate and multivariate)
• • Bayes theorem and associated algebra
• Litmus test:
• • Do you follow the derivation in R&W Ch. 2 in computing the analytic form of the posterior given a Gaussian likelihood and a Gaussian prior?

• Possible resources:

• • David Mackay's "The Humble Gaussian Distribution" tutorial: http://www.inference.org.uk/mackay/humble.pdf
• • Stanford CS229 notes on Gaussian distributions: http://cs229.stanford.edu/section/gaussians.pdf
• • Stanford CS229 notes on Gaussian processes: http://cs229.stanford.edu/section/cs229-gaussian_processes.pdf

First-order gradient-based optimization

• Key concepts:
• • Gradient descent
• • Learning rates
• • Difference between convex and non-convex functions for minimization
• Litmus test:
• • Could you fit a linear regression model via gradient descent? (see notebook below).
• Possible resources:
• • Convex Optimization overview for Stanford CS229: http://cs229.stanford.edu/section/cs229-cvxopt.pdf
• • Jupyter notebook on 'Linear Regression with NumPy' (fits linear model with gradient descent): https://www.cs.toronto.edu/~frossard/post/linear_regression/

Linear algebra

• Key concepts:
• • matrix multiplication
• • matrix inversion
• • 'least-squares' closed-form solution to linear regression
• Litmus test:
• • Could you turn the pseudocode from R&W Chapter 2 on GPs for Regression into Python code (as in HW1)? If so, you probably have all the necessary background
• Possible resources:
• • Goodfellow et al's chapter on Linear Algebra: http://www.deeplearningbook.org/contents/linear_algebra.html
• • Immersive Linear Algebra: http://immersivemath.com/ila/
• • Essence of Linear Algebra videos: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab
• • 'Computational Linear Algebra for Coders' course by fast.ai: https://github.com/fastai/numerical-linear-algebra/blob/master/README.md

Basic supervised machine learning methods

• Key concepts:
• • Linear regression
• • Logistic regression

Deep Learning Self-study Resources

Here are some related free online courses which would be good introductions to standard deep learning methods:

• FastAI's 'Deep Learning for Coders': http://course.fast.ai/
• Coursera/Andrew Ng's 'Neural Networks and Deep Learning': https://www.coursera.org/learn/neural-networks-deep-learning/home/welcome
• Stanford's CS231n "Convolutional Neural Networks for Visual Recognition": http://cs231n.stanford.edu/syllabus.html