This website covers a past offering of CS 135. For the current offering, go to https://www.cs.tufts.edu/cs/135/.

HW1: Regression

Last modified: 2026-01-22 10:20

Status: RELEASED.

Due date: Wed Jan 28, 2026 by end of day (11:59 pm ET) in Medford, MA

Jump to: Starter Code   Code Tasks   Conceptual Questions  

Overview

In this HW, you will:

  • Complete several code tasks, writing Python code based on provided starter code. You'll submit your code as .py files to the autograder link below.
  • Answer a conceptual question about your code's behavior when run on a simple data set.

Turn-in links:

Files to Turn In: Source code files are autograded. Your autograder submission should contain only these files, without any folder structure:

  • performance_metrics.py
  • LeastSquaresLinearRegression.py
  • LeastSquaresQuadraticRegression.py

Your report PDF should be no longer than 1 page and should be typeset using an application of your choosing (e.g. Word, Google Docs, Latex, etc.).

Evaluation Rubric:

  • 75% autograder score of your code
  • 20% PDF submission (results + conceptual question)
  • 5% reflection

Background

To complete this HW, you'll need some specific knowledge from the following sessions of class:

  • Linear Regression (day02)
  • Training Linear Regression (day03)

The last coding problem is related to a topic we will talk about on day04 (Polynomial feature transformations) but you do not need lecture to complete the question. We recommend working on it prior to day04 so that if you encounter questions you can ask them during lecture.

Code Tasks

Starter Code

See the hw1 folder of the public assignments repo for this class:

https://github.com/tufts-ml-courses/cs135-26s-assignments/tree/main/hw1

This starter code includes several .py files for core linear regression functionality you will need to implement, as well as a Python notebook that runs your code on a car engine dataset.

Code Task 1: Edit performance_metrics.py to implement calc_root_mean_squared_error

See the starter code here: performance_metrics.py.

Task 1 : Implement the function calc_root_mean_squared_error

See starter code for example inputs and corresponding expected outputs. Unlike in homework 0, the doctests are now in the same file, so they can run evaluated with python -m doctest performance_metrics.py

Code Task 2: Edit LeastSquaresLinearRegression.py to implement fit and predict

See the starter code here: LeastSquaresLinearRegression.py.

This file defines a LeastSquaresLinearRegressor class with the two key methods of the usual sklearn regression API: fit and predict. You will edit this file to complete the fit and the predict methods, which will demonstrate your understanding of what goes on "inside" sklearn-like regressor objects.

Task 2(a) : The fit method should take in a labeled dataset \(\{x_n, y_n\}_{n=1}^N\) (as a pair of arrays). This method will assign values for two instance attributes, stored internally in the object.

  • w_F : 1D numpy array, shape (n_features = F,)
    • Represents the 'weights'
    • Contains float64 entries of the weight coefficients
  • b : scalar float
    • Represents the 'bias' or 'intercept'.

Hint: Within a Python class, you can set an attribute like self.b = 1.0.

Nothing should be returned. You're updating the internal state of the object.

These attributes should be set using the formulas discussed in class (day03) for solving the "least squares" optimization problem (finding \(w\) and \(b\) values that minimize squared error on the training set). This code must be able to handle high-dimensional \(x_n\), so you should use the vector equation from the end of day03.

Task 2(b) : The predict method should take in an array of feature vectors \(\{x_n\}_{n=1}^N\) and produce (return) an array that stacks up the predicted responses \(\{ \hat{y}(x_n) \}_{n=1}^N\)

Recall that for linear regression, we've defined the prediction function for one instance (indexed by \(n\)) as:

$$ \hat{y}(x_n) = b + w^T x_n = b + \sum_{f=1}^F w_f x_{nf} $$

Code Task 3: Edit QuadraticRegression.py to implement non-linear regression.

So far we've only talked about linear regression. But in this problem, you'll see how you can fit a quadratic (i.e. a parabola) to data using trickery and sleight of hand. This topic will be covered in lecture on day04, but you can get started on it now (+ come to lecture with questions!).

The homework 1 notebook loads and visualizes a dataset which contains the horesepower, weight, engine displacement, and miles per gallon of 192 car engines. For now, we're interested in the relationship between an engine's horsepower and its miles per gallon. Following through the notebook, you should observe a seemingly-nonlinear relationship between these variables.

If you fit a linear regression model to a dataset with 2 features \(x_1\) and \(x_2\), you will obtain a model of the form

$$\hat{y}(x) = w_1x_1+w_2x_2+b$$

where \(w_1,w_2,b\) are all obtained via the least squares solution you implemented in task 2. The .fit() method does not care about what the features are or how they relate to each other, it just needs 2 columns \(x_1\) and \(x_2\). So we can trick it by setting \(x_1\) to be the engine's horsepower and \(x_2\) to be the square of the engine's horsepower. If we do this, the learned model will be

$$\hat{y}(x) = w_1\cdot \textnormal{horsepower}+w_2\cdot\textnormal{horsepower}^2+b$$

This is a parabola! The linear regression model does not know it is a parabola, because it only sees two variables \(x_1\) and \(x_2\) and does not realize they're secretly related to each other.

Your task is to implement a Python class that fits a parabola to a dataset with 1 input feature x_N1 and 1 target variable y_N1. This requires implementing two methods in LeastSquaresQuadraticRegression.py:

Task 3(a) : The fit method should take in a labeled dataset \(\{x_n, y_n\}_{n=1}^N\) (as a pair of arrays) where only 1 input feature is specified (i.e. each \(x_n\) is a 1-d array). This method should then add a second feature equal to the square of the provided feature, and call LeastSquaresLinearRegression.py's fit method on the resulting 2-feature dataset. As in 2(a), this method will not return anything, but will set the model's self.b and self.w_F attributes.

Hint: Calling super().fit allows the linear regression model code to set the attributes of the quadratic regression model. The starter notebook helps you visualize the parabola you fit, which can be used to check if your code is working.

Note: for the test cases to run properly, put the quadratic feature after the linear features. So your \(w_F\) should be 2 numbers, where the first is the weight on the "raw" feature and the second is the weight on the squared feature.

Task 3(b) : The predict method should take in an array \(\{x_n\}_{n=1}^N\) and produce (return) an array that stacks up the predicted responses \(\{ \hat{y}(x_n) \}_{n=1}^N\). These predictions can be computed via

$$\hat{y}(x) = w_1 \cdot x_{n1} + w_2 \cdot x_{n1}^2 + b$$

Conceptual Question

A starter notebook and car engine dataset can be found in the homework 1 directory of the public assignments repo.. Using the code you've implemented and this notebook, compute the following performance metrics:

  • Fit a linear model to predict miles per gallon (\(y\)) from horsepower (\(x\)). Then compute the root mean squared error (rMSE) of this predictor on the training data.
  • Fit a quadratic model to the same data, and compute the rMSE.

Make note of which model gets better rMSE.

NOTE: the two models you've implemented expect slightly different inputs. The linear regression model expects \(x_{NF}\) to be 2-dimensional, even if the number of features is 1. The quadratic regression model expects \(x_N\) to be 1 dimensional.

We then want you to consider a different regression task: predicting engine displacement (\(y\)) from the same predictor, horsepower (\(x\)). This relationship looks pretty linear (see notebook) compared to the previous relationship.

  • Fit a linear model to predict displacement from horsepower, and compute the rMSE of that predictor on training data.
  • Fit a quadratic model to the same data, and compute the rMSE.

Note again which of these two models gets better rMSE.

Report Task 1(a) Provide the 4 rMSE values you computed above in a typeset 2 by 2 table. There should be one column for each model (column 1 is linear, column 2 is quadratic) and one row for each regression task. Please provide values up to 2 decimal places.

Report Task 2(a) Answer the following conceptual question:

The relationship between displacement and horsepower looks pretty linear, if you ask me, so you might have been surprised to discover that the quadratic model achieved better rMSE on training data. Why is that the case? In particular, do you expect the quadratic model to always get better rMSE than the linear model on training data? Justify your answer.