Syllabus

Course Overview

The emerging research area of Bayesian Deep Learning seeks to combine the benefits of modern deep learning methods (scalable gradient-based training of flexible neural networks for regression and classification) with the benefits of modern Bayesian statistical methods to estimate probabilities and make decisions under uncertainty. The goal of this course is to bring students to the forefront of knowledge in this area through math and coding exercises, student-led discussion of recent literature, and a 2-month self-designed team research project. By completing this course, students will gain experience with designing, implementing, and evaluating new contributions in this exciting research space.

Covered topics include:

• key modeling strategies (e.g. models for functions, deep generative models)
• learning and estimation paradigms: MCMC and variational inference
• probabilistic programming platforms (we focus on JAX; we may briefly contrast alternatives like PyTorch, Tensorflow, PyMC).

Objectives

After completing this course, students will be able to:

• read a new published paper within the field and identify its contributions, strengths, and limitations
• implement a presented method in Python and apply it to an appropriate dataset
• suggest new research ideas and appropriate experiments for evaluation

Prerequisites

This course intends to bring students near the current state-of-the-art. An ambitious final project could represent a viable submission to a workshop at a major machine learning conference such as NeurIPS or ICML

To achieve this objective, we expect students to be familiar with:

• Coding in Python with modern open-source data science libraries, such as:

  • Basic array operations in numpy (computing inner products, inverting matrices, etc.)
  • Making plots in matplotlib
  • Training basic classifiers (like LogisticRegression) in scikit-learn, including appropriate hyperparameter tuning to avoid overfitting.

• Probability theory

  • you could explain the difference between a probability density function and a cumulative distribution function
  • you can mathematically define an expectation of a discrete or continuous random variable
  • you are comfortable enough with the PDF of a multivariate Gaussian that you could implement it in NumPy

• First-order gradient-based optimization

  • you could code up a simple gradient descent procedure in Python to find the minimum of f(x) = x^2

• Linear algebra

  • you could write the closed-form solution of least squares linear regression using basic matrix operations (multiply, inverse)

• Mathematical understanding of supervised machine learning methods

  • you can write down the training optimization objective for L2-penalized linear regression or logistic regression

Practically, this means having successfully completing ideally both of these Tufts courses (or their equivalents at other schools):

• CS 136 Statistical Pattern Recognition
• CS 135 Introduction to Machine Learning and/or CS 137 Deep Neural Networks

With instructor permission, diligent students who are lacking in a few of these areas will hopefully be able to catch-up on core concepts via self study and thus still be able to complete the course effectively. Please see the community-sourced Prereq. Catchup Resources Page for a list of potentially useful resources for self-study.

Enrolling and Wait Lists

As of the start of semester, we expect to have about 40 students enrolled in the course. We are currently at capacity, but some students may drop the course and leave openings for others (usually we see 5-10 openings in the first week of classes as schedules shift).

Our top priority is to provide each enrolled student with our full support, including the ability to get prompt answers to questions on Piazza and in office hours as well as the ability to get high-quality feedback on submitted homeworks, exams, and projects in a timely manner.

We understand some students are on the wait list (either formally on the wait list on SIS system, or just conceptually would like to be in the course). It is possible that students currently on the wait list may be added, but only if there is adequate staff support.

Prof. Mike Hughes will make the final decision about all wait list candidates by end of the day on Tues 1/20.

To be considered for enrollment, you must:

• Complete and submit Welcome Survey by end of day Fri 1/16

  • Explain your current state within your degree program (e.g. sophomore undergraduate in CS, Ph.D. student in Math)
  • Explain why taking the course this semester would be important to you.

Class Format for Spring 2026

This course will be offered in-person.

The first half of the course will have a mix of instructor-produced "lecture" content and interactive activities.

The second half of the course focuses on an immersive team project, and thus regular attendance is essential to keeping your team on track.

Coursework and Deliverables

There are four primary tasks for students throughout the course:

• 16% : Complete 4 homework assignments

  • Assignments and instructions: assignments.html#homeworks.
  • Short PDF reports will be turned in via Gradescope

• 20% : Complete 4 in-class quizzes that follow each homework

  • Quiz will be in class for ~20 minutes, the Tuesday after each homework is due.
  • Will cover math and core concepts behind the homework assignment

• 10% : Lead one in-class discussion

  • Each project team will be assigned to lead a class session on a topic relevant to their project
  • Select relevant paper for the class to read ahead of time
  • Meet with instructor during class or office hours to discuss strategy at least 24 hours ahead.
  • Prepare a 25 minute summary presentation to share with the class

• 54% : Team research project

  • 3% of project: Initial pitch document and follow-up meeting
  • 6% of project: Checkpoint 1 report
  • 6% of project: Checkpoint 2 report
  • 85% of project: Final report

Throughout, our evaluation will focus on your process. We wish to train you to thinking scientifically about problems, think critically about strengths and limitations of published methods, propose good hypotheses, and confirm or refute theories with well-designed experiments. It doesn't matter too much if your proposed idea works or doesn't work in the end, just that you understand why.

Grading

Final grades will be computed based on a numerical composite score that averages the 4 deliverable categories described above, using the listed percentages above as weights.

We will further enforce that the overall numerical score, on the 0.0 to 1.0 scale, cannot be more than 0.1 higher than the minimum of

• the second highest quiz grade
• the final project report grade

When assigning grades given a final numerical score (from 0.0 to 1.0), the following scale will be used:

• 0.94-1.00 : A
• 0.90-0.94 : A-
• 0.87-0.90 : B+
• 0.83-0.87 : B
• 0.80-0.83 : B-
• 0.77-0.80 : C+
• 0.73-0.77 : C
• 0.70-0.73 : C-
• 0.67-0.70 : D
• 0.63-0.67 : D
• 0.60-0.63 : D-
• Below 0.60: F

The highest possible grade of "A+" will be awarded at the instructor's discretion.

We do not round up grades. This means you must earn at least an 0.83 (not 0.825 or 0.8295 or 0.8299) to earn a B.

Collaboration Policy

Our ultimate goal is for each student to fully understand the course material. With this goal in mind, we have the following policy:

For quizzes, all work should be done individually, with no collaboration with others whatsoever.

For homeworks (HWs), we have the following policy:

You must write the text of anything that will be turned in -- all code and all written solutions -- on your own without help from other human or artificial agents. You may not directly share any code or solutions with others, regardless of if they are enrolled in the class or not.

We do encourage high-level interaction with your classmates. After you have spent at least 10 minutes thinking about the problem on your own, you may verbally discuss assignments with others in the class. You may work out solutions together on whiteboards, laptops, or other media, but you are not allowed to take away any written or electronic information from joint work sessions with others. No notes, no diagrams, and no code. Emails, text messages, and other forms of virtual communication also constitute “notes” and should not be used preparing solutions.

When preparing your solutions, you may seek general background knowledge from textbooks, materials on the course website, or existing pages on the web. If you find material on a substantially similar problem to your homework, you cannot use that material; you should instead work out your own solution. Released solutions from past years of this course are generally not available to currently enrolled students; released solutions should never be used in preparation of your homework, however you find them.

If general-purpose material was helpful to you, please cite it in your solution within your "Collaboration Statement".

You cannot ask for direct answers to homework questions via any generative AI or question answering websites such as (but not limited to) ChatGPT, Quora, StackOverflow, etc.

For projects, we have the following policy:

• The high-level plan and vision for the project should be yours. The core logic of BDL course-relevant functionality should be written by your team without help from others or GenAI.
• You are expected to fully understand and be responsible for any code you use or text that you turn in.
• You may use generative AI technologies to "automate the boring stuff" in terms of code development (e.g. helping you create beautiful plots in matplotlib).
• For the written report, you should write every word of your report yourself (no AI-assisted writing of first drafts; minor tweaks for readability are OK).
• Your report should disclose all steps that involved substantial AI assistance in its Collaboration Statement.

Academic Integrity Policy

This course will strictly follow the Academic Integrity Policy of Tufts University. Students are expected to finish course work independently when instructed, and to acknowledge all collaborators appropriately when group work is allowed. Submitted work should truthfully represent the time and effort applied.

Please refer to the Academic Integrity Policy at the following URL: https://students.tufts.edu/student-affairs/student-life-policies/academic-integrity-policy

Accessibility

Tufts and your instructor strive to create a learning environment that is welcoming students of all backgrounds.

Please see the detailed accessibility policy at the following URL: https://students.tufts.edu/student-accessibility-services

If you have any issues, please talk to your instructor so we can work to make things better. If you feel uncomfortable talking to members of the teaching staff, consider reaching out to your academic advisor, the department chair, or your dean.