Schedule

out:

- HW0

Readings:

- Course syllabus

Course Overview

Slides:

- Why take this course?

- How will semester go?

Lab Notebook:

- Intro to Arrays in NumPy [colab]

- Intro to Pandas and Plotting [colab]

due on Mon

- HW0

Readings:

- Install your Python environment

-

- Read ISLP Ch. 1 :

- Focus: 'Notation and Simple Matrix Algebra'

-

- Read ISLP Ch. 2 : Sec. 2.1 & Sec. 2.2

- Focus: 'Parametric Methods'

- Focus: 'Assessing Model Accuracy'

- Focus: 'K-Nearest Neighbors'

Regression basics

Slides:

- Regression basics

- HW0 tips / doctest guide

Lab Notebook:

- Regr Demo on Abalone [colab]

- Alt. intro to supervised ML: SML Sec. 2.1

- More on k-NN: SML Sec. 2.2: k-NN

out

- HW1

Readings:

- SML Sec. 3.1

- Notes on estimating coefs [PDF]

Linear regression

Slides:

- Linear regression

- HW0 recap

Lab Notebook:

- Linear Regr. from 1D to any-D [colab]

- Derivation: SML Sec. 3.A

- Read MML Textbook Ch. 9 : Sec. 9.1-9.2

- Derivation with probabilistic perspective

Readings:

- Skim ISLP Ch. 3 : - Focus: 3.3.2 Extensions of the Linear Model

- Esp. pages 98-99 on 'Non-linear Relationships'

- Read ISLP Ch. 5 : - Focus: 5.1.1 Validation Set Approach

- Focus: 5.1.3 k-fold Cross-Validation

Model selection & Cross validation

Slides:

- Hyperparam. Selection

Lab Notebook:

- Hyperparam Selection [colab]

Readings:

- Read SML Sec. 3.3

- Read SML Sec. 5.3

Regularization

Slides:

- Regularization

- HW1 tips

Lab Notebook:

- Regularization: L2 (ridge) vs L1 (lasso) [colab]

- Read: ISLP Ch. 6 : Sec. 6.2.1, 6.2.2, and 6.2.3

-

- Video by Prof. A. Ihler (UC-Irvine): Regularization for Linear Regression

due:

-HW1

Readings:

- Read SML Sec. 5.4

--- Focus: 'Gradient Descent' in Alg. 5.1

Gradient Descent

Slides:

- Gradient Descent

Lab Notebook:

- Gradient Descent in 1D [colab]

- Read DL Textbook Sec. 4.3 : Gradient Descent

out:

-HW2

Readings:

- Read SML Sec. 3.2

- --- Focus: 'A Statistical View of the Classification Problem

- --- Focus: 'The Logistic Regression Model for Binary Classification'

- Read SML Sec. 2.2: k-NN

- --- Focus: Example 2.3 on k-NN classification

Classification basics

Slides:

- Classifier basics

Lab Notebook:

- Binary Classification [colab]

out:

-ProjA

Readings:

- Read SML Sec. 4.6

- --- Focus: 'The Confusion Matrix and the ROC Curve'

- --- Focus: 'The F1 Score and the Precision–Recall Curve'

Evaluating Classifiers

Slides:

- Evaluating classifiers

Lab Notebook:

- Evaluating Binary Classifiers [colab]

Readings:

- Read SML Sec. 3.2

- --- Review: 'The Logistic Regression Model for Binary Classification'

- --- Focus: Training the Logistic Regression Model by Maximum Likelihood

- --- Focus: Predictions and Decision Boundaries

Logistic Regression

Slides:

- Log loss & Logistic Regression

Math/Concept Exercises:

- Work on Problem 2 of HW2

due:

-HW2

Readings:

- Read SML Sec. 3.2

- --- Focus: 'Logistic Regression for More Than Two Classes'

- Read SML Sec. 5.6

Multi-class Logistic Regr.

Slides:

- Multi-class Classifiers

- Bag-of-Words

Lab Notebook:

- Bag of Words [colab]

Readings:

- Read SML Sec. 6.1

- --- Focus: 'Two-Layer Neural Network'

- --- Focus: 'Deep Neural Network'

- --- Focus: 'Neural Networks for Classification'

Neural Net basics

Slides:

- Neural Net Basics

Math/Concept Exercises:

- Universal Approx. Demo

Lab Notebook:

- Neural Nets: MLP for XOR [colab]

Readings:

- Read SML Sec. 6.2

- --- Focus: 'Backpropagation'

- --- Focus: 'Algorithm 6.1 and Example 6.2'

Training Neural Nets

Slides:

- Forward prop & Back prop

Math/Concept Exercises:

- Backprop Visualized, from Google's ML Crash Course

Lab Notebook:

- Forward and Backward for MLPs [colab]

Readings:

- n/a

Midterm Review / Proj. A Work Day

due:

-ProjA

Readings:

- Read SML Sec. 5.5

- --- Focus: 'Stochastic Gradient Descent'

- --- Skim: 'Learning Rate and Convergence for Stochastic Gradient Descent'

- --- Skim: 'Adaptive Methods'

Stochastic Gradient Descent

Slides:

- L-BFGS and SGD

Lab Notebook:

- SGD for Linear Regr. & MLPs [colab]

- Read SML Sec. 5.4

- --- Focus: 'Second order Gradient Methods'

Readings:

- SML Sec. 6.3

- --- Focus: 'Convolutional Layer'

- --- Focus: 'Pooling Layer'

Neural Nets 2

Slides:

- From MLP to CNN and ResNet

Math/Concept Exercises:

- start problem 4 of HW3

MIDTERM EXAM

Readings:

- SML Sec. 2.3

- --- Focus: Example 2.5

- --- Focus: Example 2.6

Decision Trees

Slides:

- Decision tree basics

Lab Notebook:

- Decision Trees [colab]

due:

- HW3

Readings:

- Read SML Sec. 7.1 on Bagging

- Read SML Sec. 7.2 on Random Forest

- Skim SML Sec. 7.4 on Boosting

Ensembles

Slides:

- Ensembles of trees

Lab Notebook:

- Random Forest [colab]

<--- No Class ---> : Tufts Fri on Tue

Readings:

- Sec. 1 of Foundation Models paper

- --- Sec. 1.1: Emergence and homogenization

- --- Sec. 1.2: Social impact and the foundation models ecosystem

- --- Sec. 1.3: The future of foundation models`

Foundation models

Slides:

- Foundation models

- Stochastic Parrots paper by Bender, Gebru et al.

Readings:

- Read SML Sec. 8.1

- Read SML Sec. 8.2

Kernel Methods

Slides:

- Kernels for Regression and Classification

Lab Notebook:

- Kernels for Regr. and Clf. [colab]

due:

-HW4

Readings:

- Read SML Sec. 8.5

Support Vector Machines

Slides:

- SVMs

Lab Notebook:

- SVM demo [colab]

Readings:

- Read SML Sec. 10.4

- --- Skim: 'Auto-encoders'

- --- Focus: 'Principal Component Analysis' in SML 10.4.2 PCA

Principal Component Analysis (PCA)

Slides:

- PCA

Lab Notebook:

- PCA demo [colab]

<--- No Class --->: Thanksgiving

out:

- HW5

Readings:

- Read Koren et al. '09

Recomender Systems

Slides:

- Recommender Systems

Math/Concept Exercises:

- start HW5

- Read recent survey on Deep Learning Based Recommender System

due:

- ProjB

Readings:

- Read SML Ch. 11.2

Project B work-day

Slides:

- Proj B tips

Readings:

- Read SML Ch. 12 : Ethics in ML

- --> Focus on Sec. 12.1 'Fairness and Error Functions'

- --> Focus on Sec. 12.2 'Misleading Claims about Performance'

Fairness

Slides:

- Fairness

Math/Concept Exercises:

- Credit Score Fairness Demo

due:

- HW5

Final Exam Review

Slides:

- Final Review

Math/Concept Exercises:

- practice exam on Piazza

FINAL EXAM