Last modified: 2026-04-11 10:44
Status: RELEASED.
Due date: Wed Apr 15, 2026 by end of day (11:59 pm ET) in Medford, MA
Turn-in links:
- PDF report turned in to: https://www.gradescope.com/courses/1220989/assignments/7965444
- Finally, complete your reflection here: https://www.gradescope.com/courses/1220989/assignments/7965475
Overview
This homework consists entirely of conceptual questions covering this week's content:
- Problem 1: Conceptual questions about recommender systems
- Problem 2: Conceptual questions about PCA
Evaluation Rubric
The worth of each problem is
- 95% PDF report
- 5% reflection
See the PDF submission portal on Gradescope for the point values of each PDF subproblem. Generally, tasks with more effort will earn more potential points.
Files to Turn In:
PDF report:
- Prepare a short PDF report (no more than 3 pages).
- This document will be manually graded.
- Can use your favorite report writing tool (Word or G Docs or LaTeX or ....)
- Should be human-readable. Do not include code. Do NOT just export a jupyter notebook to PDF.
- Should have each subproblem marked via the in-browser Gradescope annotation tool
Problem 1: Recommender Systems
To complete this problem, you'll need knowledge from day21 (Recommender Systems). Your answer lengths may vary, but each question should be answerable in a single paragraph.
Conceptual Question 1a in Report
You are building a latent factor model for a collaborative filtering task. You've already taken the user-item rating values that you know and split them into train, validation, and test sets.
Now to initialize your model you need to decide what value of \(K\) to use, where \(K\) is the dimension of each embedding vector (also sometimes referred to as the number of latent factors). What would happen if you set \(K\) too large? What if you set \(K\) too small?
Conceptual Question 1b in Report
(continued problem setting from 1a) You pick a value of \(K\), but then you realize you need to initialize the values in the \(U\) and \(V\) matrices before starting stochastic gradient descent. In your first attempt, you initialize all values in both matrices to be 0s. That attempt fails wildly.
Why does this initialization not work? What needs to be changed about this initialization to make it work?
Conceptual Question 1c in Report
In lecture, when talking about content-based and collaborative filtering methods, we always assumed that our user population was fixed. But in real life, new users enter the system all the time!
Describe the differences between collaborative filtering methods and content-based filtering methods in how they would have to handle new users. Make sure to mention at least one advantage of each approach, compared to the other.
Conceptual Question 1d in Report
In practice, it is common to introduce per-user and per-item bias terms to a latent factor collaborative filtering model. That is, the collaborative filtering model becomes
Where \(b_i\) and \(c_j\) are learned (scalar) parameters, with one parameter \(b_i\) being learned for every user and one parameter \(c_j\) being learned for every item.
Describe why adding these parameters might improve the trained model's performance. Your description can be abstract, or can describe a specific scenario where introducing these parameters has a large impact on the model's performance.
Problem 2: Principal Component Analysis
To complete this problem, you'll need knowledge from day22 (PCA). Your answer lengths may vary, but each question should be answerable in a single paragraph.
Conceptual Question 2a in Report
Your friend has heard of another dimensionality reduction algorithm and found that when running it with embedding dimension \(K=2\), they get a lower reconstruction error than PCA with \(K=2\). They say you should throw away PCA and use the same algorithm as them.
Why is there still value in using PCA here?
Conceptual Question 2b in Report
After having coded a \(K=2\) PCA model, you decide you want to see what the \(K=1\) PCA model of the same dataset looks like. What additional computation is needed to obtain the \(W\) and \(z_n\) of the new PCA model?
Conceptual Question 2c in Report
Is PCA sensitive to outliers? In particular, if you add a single extreme outlier to your dataset, can it have a large impact on the low-dimensional embeddings \(z_i\) you get for other data points? Describe what impact it can have on other points' embeddings.
Hint: For problems 2c and 2d, note that you can think about principal components multiple ways. The first principal component is:
- The direction with the highest variance (we don't exactly define what this means mathematically)
- The eigenvector of the covariance matrix corresponding to the largest eigenvalue
- The direction that results in the smallest mean reconstruction error (out of all possible directions) if you project your (centered) dataset onto a line through the origin traveling in that direction.
Conceptual Question 2d in Report
Figure 1: A toy dataset.
Consider the dataset above. You want to use PCA to reduce it to 1 dimension, but you've made a mistake: you forgot to subtract the dataset mean!
You were supposed to construct the centered dataset \(X' = X - \bar{x}\) and compute the eigenvalues/eigenvectors of the covariance matrix \(S = \frac{1}{N} X'^TX'\). But instead, you computed the eigenvalues and eigenvectors with the raw, uncenetered dataset \(S^{\textnormal{WRONG}} = \frac{1}{N} X^TX\). As a result, the \(W\) you ended up with is the optimal direction to project your raw dataset onto, instead of being the optimal direction to project your centered dataset onto.
Given this mistake, what is the first (and only) principal component you find? How does that compare to the principal component you would have found if you had centered your dataset properly?