Schedule (9am)

out:

- HW0

Readings:

- Syllabus

- Install your Python environment

Course Overview

Slides:

- day01: How will this semester go?

Lab Notebook:

- Intro to Arrays in NumPy [colab]

- Intro to Pandas and Plotting [colab]

- J. van der Plas' Whirlwind Tour of Python

- B. Solomon's Guide to Vectorization in NumPy

Readings:

- Read ISLP Ch. 1 :

- Focus: 'Notation and Simple Matrix Algebra'

-

- Read ISLP Ch. 2 : Sec. 2.1 & Sec. 2.2

- Focus: 'Parametric Methods'

- Focus: 'Assessing Model Accuracy'

- Focus: 'K-Nearest Neighbors'

Regression basics

Slides:

- Regression basics

- Regression basics (annotated)

Lab Notebook:

- Regr Demo on Abalone [colab]

Notes:

- Day02 Notes

- Alt. intro to supervised ML: SML Sec. 2.1

- More on k-NN: SML Sec. 2.2: k-NN

due:

- HW0

out:

- HW1

Readings:

- SML Sec. 3.1

Linear regression

Slides:

- Linear regression

- Linear regression (annotated)

Lab Notebook:

- Linear Regr. in 1D and any-D [colab]

Notes:

- Day03 Notes

- Derivation: SML Sec. 3.A

- Read MML Textbook Ch. 9 : Sec. 9.1-9.2

- Derivation with probabilistic perspective

Readings:

- Skim ISLP Ch. 3 :

- Focus: 3.3.2 Extensions of the Linear Model

- Esp. pages 98-99 on 'Non-linear Relationships'

- Read ISLP Ch. 5 :

- Focus: 5.1.1 Validation Set Approach

- Focus: 5.1.3 k-fold Cross-Validation

Hyperparameter selection & cross validation

Slides:

- Hyperparameters & CV

- Hyperparameters & CV (annotated)

Lab Notebook:

- Hyperparam Selection [colab]

Notes:

- Day04 Notes

due:

- HW1

out:

- HW2

Readings:

- Read SML Sec. 3.3

- Read SML Sec. 5.3

Regularization

Slides:

- Regularization

- Regularization (annotated)

Lab Notebook:

- Bias and Variance [colab]

- Regularization: L2 (ridge) vs L1 (lasso) [colab]

Notes:

- Day 05 Notes

- Read: ISLP Ch. 6 : Sec. 6.2.1, 6.2.2, and 6.2.3

-

- Video by Prof. A. Ihler (UC-Irvine): Regularization for Linear Regression

Readings:

- Read SML Sec. 3.2

- --- Focus: 'A Statistical View of the Classification Problem

- --- Focus: 'The Logistic Regression Model for Binary Classification'

- Read SML Sec. 2.2: k-NN

- --- Focus: Example 2.3 on k-NN classification

Classification basics

Slides:

- Classifier basics

- Classifier basics (annotated)

Lab Notebook:

- Binary Classification [colab]

Notes:

- Day 06 Notes

due:

- HW2

out:

- HW3

Readings:

- Read SML Sec. 4.6

- --- Focus: 'The Confusion Matrix and the ROC Curve'

- --- Focus: 'The F1 Score and the Precision–Recall Curve'

Evaluating Classifiers

Slides:

- Evaluating classifiers

- Evaluating classifiers (annotated)

Lab Notebook:

- Evaluating Binary Classifiers [colab]

Notes:

- Day 07 Notes

Readings:

- Read SML Sec. 5.4

--- Focus: 'Gradient Descent' in Alg. 5.1

Gradient Descent

Slides:

- Gradient Descent

- Gradient Descent (annotated)

Lab Notebook:

- Gradient Descent in 1D [colab]

Notes:

- Day08 Notes

- Read DL Textbook Sec. 4.3 : Gradient Descent

due:

- HW3

out:

- Proj A

Readings:

- Read SML Sec. 3.2

- --- Review: 'The Logistic Regression Model for Binary Classification'

- --- Focus: Training the Logistic Regression Model by Maximum Likelihood

- --- Focus: Predictions and Decision Boundaries

Logistic Regression

Slides:

- Logistic Regression

- Logistic Regression (annotated)

Notes:

- Day09 Notes

Readings:

- Read SML Sec. 5.6

- Read SML Sec. 3.2

- --- Focus: 'Logistic Regression for More Than Two Classes'

Hyperparameter Search + Project A

Slides:

- Hyperparameters and Project A

- Hyperparameters and Project A (annotated)

Lab Notebook:

- Bag of Words [colab]

Notes:

- Day10 Notes

Readings:

- Read SML Sec. 6.1

- --- Focus: 'Two-Layer Neural Network'

- --- Focus: 'Deep Neural Network'

- --- Focus: 'Neural Networks for Classification'

Neural Net basics

Slides:

- Neural Net Basics

Math/Concept Exercises:

- Universal Approx. Demo

Lab Notebook:

- Neural Nets: MLP for XOR [colab]

Notes:

- Day11 Notes

out:

- HW4

Readings:

- Read SML Sec. 6.2

- --- Focus: 'Backpropagation'

- --- Focus: 'Algorithm 6.1 and Example 6.2'

- Read SML Sec. 5.5

- --- Focus: 'Stochastic Gradient Descent'

- --- Skim: 'Learning Rate and Convergence for Stochastic Gradient Descent'

- --- Skim: 'Adaptive Methods'

Training Neural Nets

Slides:

- Forward prop & Back prop

- Forward prop & Back prop (annotated)

Math/Concept Exercises:

- Backprop Visualized, from Google's ML Crash Course

Lab Notebook:

- Forward and Backward for MLPs [colab]

Notes:

- Day12 Notes

Readings:

- SML Sec. 6.3

- --- Focus: 'Convolutional Layer'

- --- Focus: 'Pooling Layer'

Specialized Architectures

Slides:

- From MLP to CNN and Transformers

due:

- HW4

due Friday:

- Proj A

Readings:

- Read SML Ch. 12 : Ethics in ML

- --- Focus on Sec. 12.1 'Fairness and Error Functions'

- --- Focus on Sec. 12.2 'Misleading Claims about Performance'

Fairness

Slides:

- Fairness

- ProPublica COMPAS Analysis

Readings:

- n/a

out:

- HW5

Readings:

- n/a

Explainability

Slides:

- Review and Explainability

- Review and Explainability (annotated)

- Mid-Semester Eval

Notes:

- Day17 Notes

due:

- HW5

out:

- HW6

Readings:

- n/a

Automatic Differentiation

Slides:

- Automatic Differentiation

Lab Notebook:

- Autograd [colab]

Notes:

- Day18 Notes

- Read: Lecture on Automatic Differentiation by R. Grosse

- --- Focus on slides 1-19

out:

- Proj B

Readings:

- SML Sec. 2.3

- --- Focus: Example 2.5

- --- Focus: Example 2.6

Decision Trees

Slides:

- Decision tree basics

- Decision tree basics(annotated)

Lab Notebook:

- Tree Classifiers in sklearn [colab]

Notes:

- Day19 Notes

due:

- HW6

out:

- HW7

Readings:

- Read SML Sec. 7.1 on Bagging

- Read SML Sec. 7.2 on Random Forest

- Skim SML Sec. 7.4 on Boosting

Ensembles

Slides:

- Ensembles of trees

- Ensembles of trees(annotated

Lab Notebook:

- Bootstrap and Random Forest [colab]

Notes:

- Day20 Notes

Readings:

- Read Koren et al. '09

Recommender Systems

Slides:

- Recommender Systems and Unsupervised Learning

Notes:

- Day21 Notes

- Read recent survey on Deep Learning Based Recommender System

due:

- HW7

out:

- HW8

Readings:

- Read SML Sec. 10.4

- --- Skim: 'Auto-encoders'

- --- Focus: 'Principal Component Analysis' in SML 10.4.2 PCA

Principal Component Analysis (PCA)

Slides:

- PCA

- PCA(annotated)

Lab Notebook:

- PCA demo [colab]

Notes:

- Day22 Notes

Readings:

- n/a

Image Data and Autoencoders

Slides:

- Autoencoders and Generative Models

- Autoencoders and Generative Models (annotated)

due:

- HW8

out:

- HW9

Readings:

- n/a

Clustering

Slides:

- Clustering

Readings:

- Read SML Ch. 11.2

due:

- HW9

Readings:

- n/a

due:

- Proj B