# Schedule

Schedule might change slightly as the semester goes on. Please check here regularly and refresh the page. Look for key announcements on Piazza.

## Course Introduction

Concepts: supervised learning, unsupervised learning, difference between ML and AI

Date Assignments Do Before Class Class Content Optional Extras
Wed 09/09 day01
- HW0 out

Videos on Canvas:
Course Overview

## Unit 1: Regression

Concepts: over-fitting, under-fitting, cross-validation

Methods: Linear regression, k-NN regression

Evaluation: mean squared error, mean absolute error

Work: HW1, Quiz1

Date Assignments Do Before Class Class Content Optional Extras
Mon 09/14 day02

- Focus: 'Notation and Simple Matrix Algebra'

Read ISL Textbook Ch. 2 : Sec. 2.1 & Sec. 2.2
- Focus: 'Parametric Methods'
- Focus: 'Assessing Model Accuracy'
- Focus: 'K-Nearest Neighbors'

Videos on Canvas:
Regression Basics [slides PDF]

Wed 09/16 day03
- HW0 due
- HW1 out
Read ISL Textbook Ch. 3 : Sec. 3.1-3.3
- Focus: '3.1.1 Estimating Coefficients'
- '3.3.2 Extensions beyond Linear'
- '3.3.3 Potential Problems'

- Focus: Coefficient Estimation Derivation in Eq. 5.4 - 5.12

- Estimating coefficients in 1-dim. and many dim. [PDF]
Linear Regression [slides PDF]
Read MML Textbook Ch. 9 : Sec. 9.1-9.2
- Derivation with probabilistic perspective
Mon 09/21 day04

Read ISL Textbook Ch. 5 : Sec. 5.1
- Focus: 5.1.1 Validation Set Approach
- Focus: 5.1.3 k-fold Cross-Validation

- Focus: Case for Nested Cross-Validation
Polynomial Features, Hyperparameter Selection, and Cross-Validation [slides PDF]

Wed 09/23 day05
Read ISL Textbook Ch. 6 : Sec. 6.2.1, 6.2.2, and 6.2.3

Video by Prof. A. Ihler (UC-Irvine): Regularization for Linear Regression
Regularization for Linear Regression [slides PDF]

## Unit 2: Classification

Concepts: feature engineering, hyperparameters, numerical stability, gradient descent

Methods: Logistic regression, k-NN classification

Evaluation: ROC curves, confusion matrices, cross entropy

Work: HW2, Quiz2, ProjectA

Date Assignments Do Before Class Class Content Optional Extras
Mon 09/28 day06
- ProjectA out

Some Practical Wisdom for ProjectA: "A Few Useful Things to Know about ML" by Domingos '12
Wed 09/30 day07
- HW1 due
- HW2 out
Read ISL Textbook Ch. 4 : Sec. 4.1 - 4.2

Videos on Canvas:
Binary Classification Overview [slides PDF]
- Nearest Neighbor Methods
- Linear Methods

Mon 10/05 day08
Read EMLM Textbook Ch. 2 : Ch. 2 Evaluation Metrics

Videos on Canvas:
Evaluation of Binary Classification [slides PDF]
- False Positives vs. False Negatives
- ROC Curves
- PR Curves
- Log loss (cross entropy)

Wed 10/07 day09
- Quiz1
Read ISL Textbook Ch. 4 : Sec. 4.3 - 4.3.4

Logistic Regression [slides PDF]

## Unit 3: Neural Nets

Concepts: backpropagation, stochastic gradient descent, hyperparameter selection

Methods: multi-layer perceptrons for regression and classification

Work: HW3, Quiz3

Date Assignments Do Before Class Class Content Optional Extras
Mon 10/12 day10
Read: DL Textbook Ch 6 - especially:
- Sec 6.0 (Intro)
- Sec 6.1 (xor)
- Sec 6.2 (Learning)
- Sec. 6.3 Hidden Units

Neural Networks Overview [slides PDF]
- Multi-layer Perceptrons
- Universal Function Approximation [demo]

Wed 10/14 day11
- HW2 due

Skim for high-level understanding: DL Textbook Ch 6 :
- Sec. 6.5 Backprop
Training Neural Nets with Backprop [slides PDF]

Mon 10/19 day12

Read DL Textbook Ch. 8 Sec. 8.1 - 8.3
- Focus on 8.1.2 Surrogates and Early Stopping
- Focus on 8.1.3 Batch and Minibatch algorithms
- Focus on Sec. 8.3 - 8.3.1 Stochastic Gradient Descent

Videos on Canvas:

Wed 10/21 day13
- Sec. 11.4 Hyperparameter Search

Read DL Textbook Ch 6 : Sec. 6.2.2
- Focus on 6.2.2.3 : Softmax Units
Neural Nets In Practice [slides PDF]
- Multi-class Classification
- Early Stopping
- Hyperparameter Search

## Unit 4: Trees and Ensembles

Concepts: greedy training, bagging, boosting

Methods: decision trees, random forests, XGBoost

Work: HW4, Quiz4, ProjectB

Date Assignments Do Before Class Class Content Optional Extras
Mon 10/26 day14
- ProjectA due
- ProjectB out
Read ISL Textbook Ch. 8 : Sec. 8.1
- Focus on figures for intuition

Decision Trees for Regression and Classification [slides PDF]
- An algorithmic focus on decision trees for classification
Wed 10/28 day15
- HW3 due
- HW4 out
Read ISL Textbook Ch. 5 : Sec. 5.2
- 5.2 'The Bootstrap'
Read ISL Textbook Ch. 8 : Sec. 8.2

(No videos on Canvas today)
Random Forests [slides PDF]
- Ensembles of independent predictors

Mon 11/02 day16
Read ISL Textbook Ch. 8 : Sec. 8.2

(No videos on Canvas today)
Boosting [slides PDF]
- Boosting and XGBoost

Wed 11/04 day17
Read sklearn docs on Text Feature Extraction

- From 6.2.3 through 6.2.3.7

(No videos on Canvas today)
Project B Material: Text Representations

## Unit 5: Kernel Methods

Concepts: kernel functions

Methods: kernelized linear regression, support vector machines

Work: HW5, Quiz5

Date Assignments Do Before Class Class Content Optional Extras
Mon 11/09 day18

- Focus on Sec 9.1 - 9.2
- Skim Sec. 9.4
- Focus on Sec. 9.5 relating SVMs to Logistic Regression
Support Vector Machines [slides PDF]

Tue 11/10 (Tufts-Wed-on-Tues) day19

- Focus on Sec 9.3 (which covers Kernels for SVMs)
Kernel Methods for Regression and Classification [slides PDF]
- Skim Sec. 6.1 Dual Representations
- Focus on Sec. 6.2 Constructing Kernels
Wed 11/11
- HW4 due
NO CLASS (Veteran's Day Holiday)
Mon 11/16 day20
- Read Sec. 9.4 one-vs-one and one-vs-all

- Read Sec. 10.1 Unsupervised Learning

- Read Sec. 10.2.1 What are Principal Components?

(No videos on Canvas today)
Multi-class SVMs / Intro to Dimensionality Reduction [slides]

Wed 11/18 day21
- HW5 out
Fairness in Supervised Learning [slides]
- Breakout: Discussion Guide
- No lab materials or notebook today

## Unit 6: PCA and Recommender Systems

Concepts: dimensionality reduction, matrix factorization, recommendation systems

Methods: principal components analysis, collaborative filtering models

Work: ProjectC

Date Assignments Do Before Class Class Content Optional Extras
Mon 11/23 day22
- ProjectB due
- ProjectC out
- Read Sec. 10.2.2 Another Interpretation of PCA
- Read Sec. 10.2.3 More on PCA
- Read Rec. 10.2.4 Other Uses of PCA (then go skim 6.3.1)

Principal Components Analysis [slides PDF]

Wed 11/25     NO CLASS (Thanksgiving Holiday)
Mon 11/30 day23

Matrix Factorization for Recommender Systems [slides PDF]

Wed 12/02 day24
- HW5 due

Recommender Systems [slides PDF]

## Unit 7: ML Frontiers

Date Assignments Do Before Class Class Content Optional Extras
Mon 12/07 day25

Responsible Machine Learning [slides]
Wed 12/09

Class Review and A Look to the Frontiers of ML
- Group discussion (Ask Instructor Anything)

Mon 12/21
- ProjectC due
NO CLASS (Final Exam Period)