Comp 150FML: Foundations of Machine Learning
Department of Computer Science
Tufts University
Fall 2007

Course Web Page (this page): http://www.cs.tufts.edu/comp/150FML/

Syllabus:

An advanced course focusing on statistical foundations and algorithms for machine learning. Topics include: classification and regression problems, regularization, kernel methods, model selection, boosting, support vector machines, and unsupervised learning.

Prerequisites:

Prior expose to machine learning such as in COMP-150ML, COMP-135 and a course in probability/statistics. Or permission of instructor.

Class Times:

Tuesday, Thursday 1:30-2:45 Halligan 106

Instructor:

Roni Khardon

Textbooks

We will be mainly following [B]. The other texts can provide a different perspective on (or exposition of) similar material and can be helpful.

[B] Pattern Recognition and Machine Learning, by C. M. Bishop, Springer, 2006.
[DHS] Pattern Classification (2nd edition), by R. Duda, P. Hart, and D. Stork, John Wiley & Sons, 2001.
[HTF] The elements of Statistical Learning, T. Hastie, R. Tibshirani, J. Friedman, Springer, 2001.

Course Work and Marking

Work includes regular exercises to reinforce and further develop course material (accounting for 50% of the final grade) and an empirical project applying ideas learned to an application of student's choice (50% as well).

Tentative List of Topics and Reading Assignments

Review of topics from probability theory and algebra. Linear regression and classification. Kernel methods. Gaussian processes. Graphical models, and learning and inference algorithms for them.

Lectures Topics Comments

Lectures 1-2 Maximum likelihood and Bayessian estimates for beta, bernoulli and univaiate gaussians Chapters 1/2

Lecture 3 Linear Regression and its solutions Chapters 1/3

Lectures 4-7 Multivariate Normal Distributions, details, techniques and ML estimates Chapter 2

Lectures 8-9 Bayesian Linear Regression with priors for (w,b) Chapters 3 + extra notes

Lecture 10 Baysian Model Selection Chapter 3/4

Lecture 11 Exponential family of distributions Chapter 2/3

Lectures 12-13 Generative moedls for Classification Chapter 4

Lecture 14 Fisher's linear discriminant Chapter 4

Lecture 15-16 ML and Bayesian Logistic Regression Chapter 4

Lecture 17-18 Introuction to kernels: perceptron, nearest neighbors, least squares, and Gaussian processes Chapter 6 + extra notes

Lecture 19-21 Support vector machines and relevance vector machines Chapter 7 + extra notes

Lecture 22-23 Review of Bayes Networks and exact and approximate inference algorithms Chapters 8, 11

Lecture 24 The EM algorithm Chapter 9

Lecture 25 Infinite mixture of trees from ICML 2007.

Assignment 1 HW1.txt

Assignment 2 hw2.pdf

Assignment 3 HW3.txt

Assignment 4 hw4.pdf

Assignment 5 hw5.pdf

Lectures	Topics	Comments
Lectures 1-2	Maximum likelihood and Bayessian estimates for beta, bernoulli and univaiate gaussians	Chapters 1/2
Lecture 3	Linear Regression and its solutions	Chapters 1/3
Lectures 4-7	Multivariate Normal Distributions, details, techniques and ML estimates	Chapter 2
Lectures 8-9	Bayesian Linear Regression with priors for (w,b)	Chapters 3 + extra notes
Lecture 10	Baysian Model Selection	Chapter 3/4
Lecture 11	Exponential family of distributions	Chapter 2/3
Lectures 12-13	Generative moedls for Classification	Chapter 4
Lecture 14	Fisher's linear discriminant	Chapter 4
Lecture 15-16	ML and Bayesian Logistic Regression	Chapter 4
Lecture 17-18	Introuction to kernels: perceptron, nearest neighbors, least squares, and Gaussian processes	Chapter 6 + extra notes
Lecture 19-21	Support vector machines and relevance vector machines	Chapter 7 + extra notes
Lecture 22-23	Review of Bayes Networks and exact and approximate inference algorithms	Chapters 8, 11
Lecture 24	The EM algorithm	Chapter 9
Lecture 25	Infinite mixture of trees from ICML 2007.
Assignment 1	HW1.txt
Assignment 2	hw2.pdf
Assignment 3	HW3.txt
Assignment 4	hw4.pdf
Assignment 5	hw5.pdf

Comp 150FML: Foundations of Machine Learning Department of Computer Science Tufts University Fall 2007