Syllabus


CS 152 BDL: Bayesian Deep Learning
Department of Computer Science, Tufts University
Class Meetings for Fall 2022:
  • Mon and Wed 10:30-11:45 am ET
  • Location: Sci Tech Center Room 135, 4 Colby St, Medford, MA
Instructor: Mike Hughes, Assistant Professor of Computer Science
Grad TA: Zhe Huang, Ph.D. student, Tufts CS.
Have questions? Need help?
  • Get rapid online help via our Piazza discussion forum
  • Get in-person help at regular Office Hours
  • For personal issues, email the instructor: mhughes(AT)cs.tufts.edu

Quick Links: [Prereqs] [Wait List] [Deliverables] [Late Work Policy] [Collaboration Policy] [Grading Rubric]

Course Overview

The emerging research area of Bayesian Deep Learning seeks to combine the benefits of modern deep learning methods (scalable gradient-based training of flexible neural networks for regression and classification) with the benefits of modern Bayesian statistical methods to estimate probabilities and make decisions under uncertainty. The goal of this course is to bring students to the forefront of knowledge in this area through coding exercises, student-led discussion of recent literature, and a long-term research project. Covered topics include key modeling innovations (e.g. models for functions and deep generative models), learning paradigms (e.g. MCMC and variational inference), and probabilistic programming platforms (e.g. JAX, PyTorch, Tensorflow, PyMC3). By completing a 2-month self-designed research project, students will gain experience with designing, implementing, and evaluating new contributions in this exciting research space.

Objectives

After completing this course, students will be able to:

  • read a new published paper within the field and identify its contributions, strengths, and limitations
  • implement a presented method in Python and apply it to an appropriate dataset
  • suggest new research ideas and appropriate experiments for evaluation

Prerequisites

This course intends to bring students near the current state-of-the-art. An ambitious final project could represent a viable submission to a workshop at a major machine learning conference such as NeurIPS or ICML

To achieve this objective, we expect students to be familiar with:

  • Coding in Python with modern open-source data science libraries, such as:
    • Basic array operations in numpy (computing inner products, inverting matrices, etc.)
    • Making basic plots or grids of plots in matplotlib
    • Training basic classifiers (like LogisticRegression) in scikit-learn
  • Probability theory
    • e.g. you could explain the difference between a probability density function and a cumulative density function
  • First-order gradient-based optimization
    • e.g. you could code up a simple gradient descent procedure in Python to find the minimum of f(x) = x^2
  • Basic supervised machine learning methods
    • e.g. you can describe the difference between linear regression or logistic regression
  • Linear algebra
    • e.g. you could write the closed-form solution of least squares linear regression using basic matrix operations (multiply, inverse)

Practically, at Tufts this means having successfully completed one of:

  • COMP 135 (Introduction to Machine Learning)
  • COMP 136 (Statistical Pattern Recognition)
  • OR permission of the instructor

With instructor permission, diligent students who are lacking in a few of these areas will hopefully be able to catch-up on core concepts via self study and thus still be able to complete the course effectively. Please see the community-sourced Prereq. Catchup Resources Page for a list of potentially useful resources for self-study.

Enrolling and Wait Lists

As of the start of semester, we expect to have about 40 students enrolled in the course. We are currently at capacity, but some students may drop the course and leave openings for others (usually we see 5-10 openings in the first week of classes as schedules shift).

Our top priority is to provide each enrolled student with our full support, including the ability to get prompt answers to questions on Piazza and in office hours as well as the ability to get high-quality feedback on submitted homeworks, exams, and projects in a timely manner.

We understand some students are on the wait list (either formally on the wait list on SIS system, or just conceptually would like to be in the course). It is possible that students currently on the wait list may be added, but only if there is adequate staff support.

Prof. Mike Hughes will make the final decision about all wait list candidates by end of the second week (by Fri 9/16, well before the ADD deadline).

To be considered for enrollment, you must:

  • Complete and submit Welcome Survey by end of day Wed 9/14
    • Explain your current state within your degree program (e.g. sophomore undergraduate in CS, Ph.D. student in Math)
    • Explain why taking the course this semester would be important to you.

Class Format for Fall 2022

This course will be offered in-person, with some backup options for students to view recorded class content later via Zoom recordings.

The first half of the course will have a mix of instructor-produced "lecture" content and interactive activities.

The second half of the course focuses on an immersive team project, and thus regular attendance is essential to keeping your team on track.

Coursework and Deliverables

There are four primary tasks for students throughout the course:

  • 28% : Complete 4 homework assignments
    • Short PDF reports will be turned in via Gradescope
  • 3% : Participate in class discussions
  • 7% : Lead one in-class discussion
    • 4 student leaders will be assigned to each class from mid October onward
    • Read the paper well in advance of the assigned date
    • Meet with instructor during office hours one week before to discuss strategy
    • Prepare a 20 minute summary presentation to share with the class
  • 62% : Team research project
    • 7% Initial pitch presentation
    • 7% Checkpoint 1 report
    • 8% Checkpoint 2 report

Throughout, our evaluation will focus on your process. We wish to train you to thinking scientifically about problems, think critically about strengths and limitations of published methods, propose good hypotheses, and confirm or refute theories with well-designed experiments. It doesn't matter too much if your proposed idea works or doesn't work in the end, just that you understand why.

Late work Policy

We want students to develop the skills of planning ahead and delivering work on time. To facilitate learning, we also want to be able to release solutions quickly and discuss recent assignments soon after deadlines. On the other hand, we know that this semester offers particular challenges, and we wish to be flexible and accommodating within reason.

With these goals in mind, we have the following policy:

Each student will have 192 total late hours (= 8 late days) to use throughout the semester across all homeworks.

For each individual assignment, you can submit beyond the posted deadline at most 96 hours (4 days) and still receive full credit. Thus, for one assignment due at Tue 11:59pm ET, you could submit by Saturday at 11:59pm ET.

This late work deadline is key to our classroom goals. It allows us to:

  • always release homework solutions on Monday mornings
  • discuss the assignment in class on Mon
  • be sure we can return all graded work by end of day Wed (8 days after you turned it in)

The timestamp recorded on Gradescope will be official. Late time is rounded up to the nearest hour. For example, if the assignment is due at 3pm and you turn it in at 3:05pm, you have used one whole hour.

Beyond your allowance of 8 late days (and beyond the 4-day limit per assignment), zero credit will be awarded except in cases of truly unforeseen exceptional circumstances (e.g. family emergency, medical emergency). Students with exceptional circumstances should contact the instructor to make other arrangements as soon as possible.

Collaboration Policy

For homeworks: we encourage you to work actively with other students, but you must be an active participant (asking questions, contributing ideas) and you should write your solutions document alone. At the top of your writeup, you must include the names of any people you worked with, and in what way you worked them (discussed ideas, debugged math, team coding). We may occasionally check in with groups to ascertain that everyone in the group was participating in accordance with this policy.

For final projects: you will work in teams. Each team should submit one report at each checkpoint and will give one presentation. Each member of the team is expected to actively participate in every stage of the project (ideation, math, coding, writing, etc.). Please write all names at the top of every report, with brief notes about how work was divided among team members. Larger teams will be expected to produce more interesting content.

Grading

Final grades will be computed based on a numerical score that averages the 4 categories of coursework above, using the prov

When assigning grades given a final numerical score (from 0.0 to 1.0), the following scale will be used:

  • 0.94-1.00 : A
  • 0.90-0.94 : A-
  • 0.87-0.90 : B+
  • 0.83-0.87 : B
  • 0.80-0.83 : B-
  • 0.77-0.80 : C+
  • 0.73-0.77 : C
  • 0.70-0.73 : C-
  • 0.67-0.70 : D
  • 0.63-0.67 : D
  • 0.60-0.63 : D-
  • Below 0.60: F

The highest possible grade of "A+" will be awarded at the instructor's discretion.

We do not round up grades. This means you must earn at least an 0.83 (not 0.825 or 0.8295 or 0.8299) to earn a B.

Academic Integrity Policy

This course will strictly follow the Academic Integrity Policy of Tufts University. Students are expected to finish course work independently when instructed, and to acknowledge all collaborators appropriately when group work is allowed. Submitted work should truthfully represent the time and effort applied.

Please refer to the Academic Integrity Policy at the following URL: https://students.tufts.edu/student-affairs/student-life-policies/academic-integrity-policy

Accessibility

Tufts and your instructor strive to create a learning environment that is welcoming students of all backgrounds.

Please see the detailed accessibility policy at the following URL: https://students.tufts.edu/student-accessibility-services