Resources

Jump to: Software • Related Courses • Machine Learning at Tufts • Self-Study Resources

Software

For this course, we require programming assignments to be implemented in Python 3.6+. Using a consistent language allows us to talk about implementation details in class and makes grading solutions more consistent and time-efficient.

Students are responsible for maintaining their own software environment on their personal computer. We highly suggest that you consider the free 'conda' environment manager from Anaconda, Inc.: https://conda.io/docs/user-guide/getting-started.html

For detailed instructions, see the [Python Setup Instructions page]

Textbook toolbox

A toolbox for the PRML textbook has been implemented in the Matlab language: * https://prml.github.io/ * https://github.com/PRML/PRMLT

Does not appear to be sponsored by the textbook author, but may be a useful resource.

Related Courses

Statistical Pattern Recognition (COMP 136) at Tufts

Previous offerings:

• 2019 spring, taught by Rishit Sheth, Ph.D.
• 2017 fall and earlier, with Prof. Roni Khardon (sorry, website no longer available)

Related courses at other universities

• Probabilistic Learning: Theory and Algorithms at UC-Irvine, taught by Prof. Padhraic Smyth
• • Lots of useful lecture notes!

• Statistical Machine Learning at UMass-Amherst, taught by Prof. Justin Domke

• Foundations of Graphical Models at Columbia, taught by Prof. David Blei

Machine Learning at Tufts

For machine learning research activity at Tufts, see the ML Research Group Website:

• https://www.cs.tufts.edu/research/ml

For a recent listing of ML courses, see:

• https://www.cs.tufts.edu/research/ml/courses.html

For current ML research opportunities for students, see:

• https://www.cs.tufts.edu/research/ml/hiring.html

Self-Study Resources

Here are some useful resources to help you catch up if you are missing some of the pre-requisite knowledge. Please contribute new resources by starting a topic on the class discussion forum.

Probability

• Key concepts:
• • Sum rule and product rule of probability
• • Bayes theorem and associated algebra
• • Continuous and discrete random variables
• Litmus test:
• • Do the lecture notes from day01.pdf seem familiar? Do you have enough prevous experience and math sophistication to follow these easily?

• Possible resources:

• • Probability review notes from Prof. David Blei: http://www.cs.columbia.edu/~blei/fogm/2016F/doc/probability_review.pdf
• • Stanford CS229 notes on Gaussian distributions: http://cs229.stanford.edu/section/gaussians.pdf

First-order gradient-based optimization

• Key concepts:
• • Gradient descent
• • Learning rates
• • Difference between convex and non-convex functions for minimization
• Litmus test:
• • Could you fit a linear regression model via gradient descent? (see notebook below).
• Possible resources:
• • Convex Optimization overview for Stanford CS229: http://cs229.stanford.edu/section/cs229-cvxopt.pdf
• • Jupyter notebook on 'Linear Regression with NumPy' (fits linear model with gradient descent): https://www.cs.toronto.edu/~frossard/post/linear_regression/

Linear algebra

• Key concepts:
• • matrix multiplication
• • matrix inversion
• • 'least-squares' closed-form solution to linear regression

• Possible resources:

• • Goodfellow et al's chapter on Linear Algebra: http://www.deeplearningbook.org/contents/linear_algebra.html
• • Immersive Linear Algebra: http://immersivemath.com/ila/
• • Essence of Linear Algebra videos: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab
• • 'Computational Linear Algebra for Coders' course by fast.ai: https://github.com/fastai/numerical-linear-algebra/blob/master/README.md

Basic supervised machine learning methods

• Key concepts:
• • Linear regression
• • Logistic regression