Jump to: [Unit 1: Discrete] - [Unit 2: Regression] - [Unit 3: Mixtures] - [Unit 4: Time Series] - [Unit 5: MCMC]
For any class day with assigned readings, you should complete them before the start of class.
Schedule might change slightly as the semester goes on. Please check here regularly and refresh the page. Look for key announcements on Piazza.
Unit 1: Foundations for Discrete Data
Key ideas: Probability fundamentals, Maximum likelihood estimation, MAP estimation, Beta-Bernoulli and Dirichlet-Multinomial distributions, conjugacy, exponential family distributions
Models: Dirichlet-multinomial models for text
Math Practice: HW1, which covers joint, conditional, and marginal distributions; ML/MAP point estimation; Bayesian posterior estimation; Beta PDF and Gamma functions; conjugacy
Coding Practice: CP1 which covers text modeling with unigram distributions
| date |
assignments |
topic |
required |
optional |
|---|
| Wed 01/15 |
|
- Course Overview & Probability Refresher
-
|
>>> Focus esp. on 1.2.1, 1.2.2, and 1.2.3
|
|
| Mon 01/21 |
|
NO CLASS: MLK Holiday |
|
|
| Wed 01/22 |
- out:
-
|
- Maximum Likelihood for Binary Data
-
|
>>> Contrast maximum likelihood vs Bayesian approach
>>> Focus on ML estimator Eq. 2.5-2.8
|
|
| Mon 01/27 |
|
- Beta-Bernoulli models for Binary Data
-
|
>>> Focus on the Beta and Bernoulli distributions
|
|
| Wed 01/29 |
|
- Dirichlet-Discrete models for Count Data
- Guest Lecturer: Prof. Anselm Blumer
|
>>> Focus on the Dirichlet distribution
|
|
Unit 2: Multivariate Gaussians and Regression
Key ideas: multivariate Gaussian distributions, model selection, Laplace approximation
Models: Bayesian linear regression, Bayesian logistic regression, generalized linear models
Algorithms: gradient descent, methods for model selection
Math Practice: HW2
Coding Practice: CP2
| date |
assignments |
topic |
required |
optional |
|---|
| Mon 02/03 |
|
- Gaussians and Linear Algebra Review
-
|
>>> Focus on definitions of Univariate and Multivariate Gaussian, ML estimators
|
|
| Wed 02/05 |
- due:
-
|
- Multivariate Gaussians and Linear Algebra
-
|
>>> Dig into linear algebra of Multivariate Gaussian
|
|
| Mon 02/10 |
|
- Bayesian Linear Regression 1/2
-
|
>>> Focus on ML estimator for linear regression
|
>>> Bias/Variance tradeoff
|
| Wed 02/12 |
- due:
-
|
- Bayesian Linear Regression 2/2
-
|
>> Focus on the posterior and predictive distribution
|
|
| Mon 02/17 |
|
NO CLASS: President's Day |
|
|
| Wed 02/19 |
|
- Model Selection for Linear Regression
- Guest Lecturer: Prof. Liping Liu
|
>> Focus on model selection and hyperparameter estimation
|
|
| Thu 02/20 |
|
- Bayesian Linear Models for Classification
-
|
>>> Focus on Binary and Multi-class classification
>>> Try to understand the Laplace approximation
|
|
| Mon 02/24 |
|
- Gradient Descent for Bayesian Logistic Regression
-
|
>>> Bayesian Logistic Regression
|
|
Unit 3: Clustering, Mixture Models, and Expectation-Maximization
Key ideas: coordinate ascent optimization, local optima, expectations
Models: Mixture models
Algorithms: k-means, expectation maximization, variational inference
Math Practice: HW3
Coding Practice: CP3
| date |
assignments |
topic |
required |
optional |
|---|
| Wed 02/26 |
- due:
-
|
- K-Means Clustering
-
|
|
|
| Mon 03/02 |
|
- Gaussian Mixture Models and ML Estimation
-
|
|
|
| Wed 03/04 |
- due:
-
|
- Midterm Review
-
|
|
|
| Mon 03/09 |
|
- ML Estimation with GMMs: Expectation Maximization and Gradient Descent
-
|
|
|
| Wed 03/11 |
Take-home Midterm due 03/25 |
CANCELLED due to COVID-19 |
|
|
| Mon 03/16 |
|
NO CLASS: SPRING BREAK |
|
|
| Wed 03/18 |
|
NO CLASS: SPRING BREAK |
|
|
| Mon 03/23 |
|
CANCELLED due to COVID-19 |
|
|
| Wed 03/25 |
- out:
-
|
- Expectation Maximization for GMMs
- Video part 1 : Recap of GMMs - Video part 2 : GMMs with Latent Assignments Z - Video part 3 : Expectations of Complete Likelihood - Video part 4 : Lower Bound of Incomplete Likelihood - Video part 5 : EM as Coordinate Ascent on Lower Bound Objective
|
|
|
Unit 4: Hidden Markov models for Time-Series
Key ideas: dynamic programming, joint MAP vs. marginal MAP
Models: Hidden markov models
Algorithms: forward-backward algorithm, Viterbi algorithm, belief propagation algorithm
Math Practice: HW4
Coding Practice: CP4
| date |
assignments |
topic |
required |
optional |
|---|
| Mon 03/30 |
|
- Markov Models and Hidden Markov Models
- Video day17 part1 : Unit 4 Motivation: Dependencies in Sequential Data - Video day17 part3 : Markov models for discrete sequences
|
>>> Only the intro before 13.2.1
|
>>> For more depth and coding practice
|
| Wed 04/01 |
- due:
-
|
- EM for HMMs
- Video day18 part5 : E-step for HMMs: forward algorithm and backward algorithm
|
>>> Think about numerical stability of calculations
|
|
| Mon 04/06 |
- out:
-
|
- Viterbi for HMMs
- Video day19 part1 : Motivation for inferring hidden states - Video day19 part2 : Most likely hidden sequence problem - Video day19 part3 : Intuition behind recursive solution
|
|
|
Unit 5: Markov Chain Monte Carlo
Key ideas: Markov chains, revisibility, ergodicity, detailed balance, probabilistic programming
Models: mixture models, Bayesian logistic regression, factor analysis
Algorithms: Metropolis-Hastings algorithm, Gibbs sampling
Math Practice: HW5
Coding Practice: CP5
| date |
assignments |
topic |
required |
optional |
|---|
| Wed 04/08 |
- due:
-
|
- Sampling Methods and Markov Chain Monte Carlo
- Video day20 part1 : Monte Carlo estimates of expectations - Video day20 part2 : Directed graphical models & ancestral sampling - Video day20 part3 : MCMC Intro, Stationary Distributions, Ergodicity - Video day20 part5 : Transformations of Sampled Variables
|
>>> Focus on basic methods using transformed uniform r.v. in 11.1.1
>>> Skim rejection sampling in 11.1.2
>>> Skim importance sampling in 11.1.4
>>> Focus on the overview section and 11.2.1 Markov chains
|
|
| Mon 04/13 |
- out:
-
|
- Random Walk Proposals and Metropolis-Hastings Algorithm
- Video day21 part1 : Markov Transitions that Propose then Accept/Reject - Video day21 part3 : Detailed Balance, Proof of Random Walk's Stationary Distribution - Video day21 part4 : Metropolis and Metropolis-Hastings algorithms
|
|
|
| Wed 04/15 |
- due:
-
|
- Gibbs Sampling
- Video day22 part1 : Overview of Sampling Vector Random Var. - Video day22 part2 : Pro/con comparison of Gibbs vs. Random Walk - Video day22 part4 : Proof sketch of Gibbs Sampling correctness
|
|
|
| Mon 04/20 |
|
- NO CLASS (Patriot's Day)
BONUS CONTENT: Hamiltonian Monte Carlo
|
This is extra material. Nothing from this will be on any exam.
|
|
| Wed 04/22 |
- due:
- Quizlet4 (late this week)
|
- Probabilistic Programming
-
|
>>> Read thru Case Study 1
>>> Skim, focus on examples
|
>>> Example of Gaussian mixtures in PyMC3
|
| Mon 04/27 |
- due:
- Quizlet5
- CP5 (late deadline 5/3)
|
- Final Exam Review
-
|
|
|
| FINALS WEEK |
|
- FINAL EXAM
-
|
|
|