- Office hours on Zoom
- Contact: mhughes(AT)cs.tufts.edu for personal issues only (for almost all questions, use Piazza forums)
- Piazza Discussion Forum for COMP 136 Spring 2021
- Use discussion forums for any question about course content or assignments
Course Overview
This course provides the theoretical and computational foundations for probabilistic machine learning. The focus is on probabilistic models, which are especially useful for any application where observed data could be noisy, sometimes missing, or not available in large quantities. We emphasize representing uncertainty with formal distributions and trying to average over these distributions when making decisions (as done in the Bayesian approach).
We will study the following probabilistic models:
- unigram models for discrete word count data (e.g. next word prediction)
- regression and classification
- mixture models
- hidden Markov models and other models for sequential data
- general directed graphical models.
Algorithms studied include: gradient descent (first-order and second-order), expectation maximization, variational inference, and Markov chain Monte Carlo methods.
Objectives
After completing this course, students will be able to:
- Demonstrate formal mathematical understanding of probabilistic models.
- Given an applied data analysis task, select a relevant probabilistic model, fit the model on a relevant dataset using an appropriately chosen approximate inference method, and analyze the results.
- Analyze numerical accuracy and stability of common probabilistic ML algorithms (e.g. avoiding overflow/underflow)
- Analyze scalability considerations of common probabilistic ML methods (including runtime and memory complexity requirements).
Prerequisites
This course intends to provide students a solid foundation in statistical machine learning methods.
To achieve this objective, we expect students to be familiar with the following before taking the course:
- Probability theory
- e.g. you could explain the difference between a probability density function (PDF) and a cumulative distribution function (CDF)
- Basic linear algebra (comfort with matrix/vector notation, have at least seen inverses/determinants before)
- e.g. you could write the closed-form solution of least squares linear regression using basic matrix operations (multiply, inverse)
- First-order gradient-based optimization
- e.g. you could code up a simple gradient descent procedure in Python to find the minimum of functions like f(x) = x^2
- Basic supervised machine learning methods
- e.g. you can describe the mathematical learning objectives for linear regression and logistic regression
- Coding in Python with modern open-source data science libraries
- Basic array operations in numpy (computing inner products, inverting matrices, etc.)
- Making basic plots or grids of plots in matplotlib
- Training basic classifiers (like LogisticRegression) in scikit-learn
Practically, this means having successfully completed at least one of these courses (or their equivalent outside of Tufts):
With instructor permission, diligent students who are lacking in a few of these areas of coursework could be able to catch-up on core concepts via self study and thus still be able to complete the course effectively. Please see the community-sourced Resources Page for a list of potentially useful resources for self-study.
Enrolling and Wait Lists
As of the start of semester, we expect to have about 40 students enrolled in the course. We are currently at capacity, but some students may drop the course and leave openings for others (usually we see 5-10 openings in the first week of classes as schedules shift).
Our top priority is to provide each enrolled student with our full support, including the ability to get prompt answers to questions on Piazza and in office hours as well as the ability to get high-quality feedback on submitted homeworks, exams, and projects in a timely manner.
We understand some students are on the wait list (either formally on the wait list on SIS system, or just conceptually would like to be in the course). It is possible that students currently on the wait list may be added, but only if there is adequate staff support.
Prof. Mike Hughes will make the final decision about all wait list candidates by end of the second week (by Fri 2/12, well before the ADD deadline).
To be considered for enrollment, you must do these two things:
- Complete and submit HW0 by end of day Wed 2/10.
- This action shows you have the necessary skills and would take the course seriously
- Message the instructor by end of day Wed 2/10 via email with subject "COMP 136 Wait List Request"
- Explain your current state within your degree program (e.g. sophomore undergraduate in CS, Ph.D. student in Math)
- Explain why taking the course this semester would be important to you.
Class Format for Spring 2021
Due to the ongoing pandemic, this course will be in virtual-only format for Spring 2021 semester. Due to the large class size and the need to keep our whole community safe, all interactions will be virtual, including all in-class sessions and office hours.
See the Resources page for expectations about your computing environment. We expect we can accommodate any student who needs to complete the course in a fully remote environment, from anywhere on Earth, as long as you have reliable internet access and access to a modern desktop or laptop computer (Not just a chromebook) with at least 1GB RAM and 2.0GHz processor and ability to install our course Python environment.
If you have concerns about your computing resources being adequate , please contact the course staff via Piazza ASAP.
How to attend class
After the first week, we will expect students to be signed up on Piazza (accessible to any student either enrolled or on the waitlist).
We will post relevant links to virtual class meetings (and office hours) on Piazza Resources page
What will we do in class
Each synchronous class session will occur at the scheduled time (Mon and Wed from 430-545pm ET).
Before each class, you are expected to complete the "Do Before Class" activities posted on that day's Schedule. These will include:
- Assigned reading from the textbook (and occasionally other high-quality sources).
- Assigned video lectures (broken into 10-20 minute segments about a coherent topic)
You should also download any relevant in-class exercises or notebooks to prepare (materials will be posted on the Schedule).
In class, we will typically have the following structure, all over Zoom:
- First 5 min.: Course Announcements (instructor led)
- Next 10 min.: Recap key concepts for the day (instructor led)
- Next 30 min.: Breakout into small groups to work through exercises and solidify concepts
- Last 30 min.: Flex time. Instructors/TA available for on-demand questions. Breakout discussions can continue.
The "flex time" is set aside specifically to acknowledge that in our flipped classroom model, you spend more time than usual outside of class preparing via readings and lectures. We will not require attendance or deliver new content in this time slot, but instead offer "office hours" like on-demand support at a time that should be available in most students' schedules.
We will strive to create an exciting, highly interactive virtual classroom, with lots of opportunities for students to ask questions and get feedback from the professor, TAs, and peers.
Each student is responsible for shaping this environment: Please participate actively and respectfully! Show up regularly, turn on your camera and your audio, and get engaged.
Attendance
Participation in class is strongly encouraged, as you will get hands-on practice with material and have a chance to ask questions of the instructor and TAs, as well as your peers.
We do not require attendance at any class or track attendance. We understand some of you may be in remote parts of the world where it is difficult to attend every class. We hope you make the effort, but we designed the course so that you can succeed even if you miss a meeting or two.
Instructional material (readings, notes, and videos) will always be "prerecorded" and released on the Schedule page in advance, under "Do Before Class".
We will record video and audio for the main track of each interactive class session to capture important announcements and highlight key takeaways. releasing that video within 24 hours to the Piazza resources page. If you do not wish to be recorded, please do not participate in this session (silently listen with microphone and camera off).
The most valueable learning interactions will occur in breakout rooms. These cannot and will not be recorded.
We do count a small part of a student's grade as participation, which are measured in two ways:
- being regularly active in Piazza forum discussions (posting 8 or more times throughout the semester)
- being regularly active in live class discussions (being a memorable participant in 8 or more meeting breakouts throughout the semester)
What will we do outside of class?
The course is organized into 5 topical units.
Each of the 5 units will have the following assigned work outside of class:
- 1 written homework (HW), to build math skills (derivation and analysis, resulting in a LaTeX typeset report)
- 1 coding practical (CP), to build implementation skills (auto-graded Python exercises + a short report of figures and analysis)
- 1 short take-at-home quiz (time limit 30 minutes, handwritten and submitted to gradescope)
At the end of each topical unit, the short quiz will verify your understanding of main concepts (vast majority of questions will be math or conceptual questions, occasionally there might be a programming question). These quizzes are designed to be completable within 15 minutes by most students.
For a complete list of graded assigned work, see the Assignments page.
Note that there is a special "first homework" (HW0) designed to make sure you have necessary prerequisite knowledge and that you become familiar with LaTeX for math report preparation.
Late work Policy
We want students to develop the skills of planning ahead and delivering work on time. To facilitate learning, we also want to be able to release solutions quickly and discuss recent assignments soon after deadlines. On the other hand, we know that this semester offers particular challenges, and we wish to be flexible and accommodating within reason.
With these goals in mind, we have the following policy:
Each student will have 240 total late hours (= 10 late days) to use throughout the semester across all homeworks and coding practicals.
For each individual assignment, you can submit beyond the posted deadline at most 96 hours (4 days) and still receive full credit. Thus, for one assignment due at Wed 11:59pm ET, you could submit by the following Sunday at 11:59pm ET.
This late work deadline is key to our classroom goals. It allows us to always release homework solutions on Monday mornings a few days before the required quiz on that unit is due, and lets us discuss the assignment in class on Monday afternoon without issue.
The timestamp recorded on Gradescope will be official. Late time is rounded up to the nearest hour. For example, if the assignment is due at 3pm and you turn it in at 3:05pm, you have used one whole hour.
Beyond your allowance of 10 late days, zero credit will be awarded except in cases of truly unforeseen exceptional circumstances (e.g. family emergency, medical emergency). Students with exceptional circumstances should contact the instructor to make other arrangements as soon as possible.
Exams
There will be two formal take-home exams, a midterm and a final exam, each lasting less than 2 hours. See the Schedule for the specific dates and times.
The exams will test the key concepts covered up to that point in the course, with mostly a focus on mathematical analysis skills but perhaps a computational question or two.
The short quizzes assigned after each unit should be considered good preparation for the kinds of questions that might appear on an exam.
Workload
Each week, you should expect to spend about 8-12 hours on this class.
Here's our recommended break-down of how you'll spend time each week:
- 1.75 hr / wk preparation before Mon class (reading, lecture videos)
- 1.00 hr / wk active participation in Mon class (excludes flex time)
- 1.75 hr / wk preparation before Wed class (reading, lecture videos)
- 1.00 hr / wk active participation in Wed class (excludes flex time)
- 4.00 hr / wk on HW or CP assignment (one assignment due each week, takes ~4 hr total)
- 0.50 hr / wk preparing for and taking quiz (quizzes happen every few weeks, so get 0.5 hr prep plus 0.5 time taking the quiz)
This totals to ~10.00 hr / wk.
Grading
Final grades will be computed based on a numerical score via the following weighted average:
- 25% homeworks (2% HW0, 23% evenly averaged across HW1-HW5)
- 25% coding practicals (one per unit, evenly averaged across all 5 units)
- 19% short quizzes (one per unit, averaged after dropping the lowest quiz)
- 15% midterm exam score (a 1 week take-home exam)
- 15% final exam score (a 1 week take-home exam)
- 1% participation on Piazza and in class
When assigning grades given a final numerical score (from 0.0 to 1.0), the following scale will be used:
- 0.94-1.00 : A
- 0.90-0.94 : A-
- 0.87-0.90 : B+
- 0.83-0.87 : B
- 0.80-0.83 : B-
- 0.77-0.80 : C+
- 0.73-0.77 : C
- 0.70-0.73 : C-
- 0.67-0.70 : D
- 0.63-0.67 : D
- 0.60-0.63 : D-
- Below 0.60: F
The highest possible grade of "A+" will be awarded at the instructor's discretion.
We do not round up grades. This means you must earn at least an 0.83 (not 0.825 or 0.8295 or 0.8299) to earn a B.
Textbook
As a primary textbook, we will use "Pattern Recognition and Machine Learning" by Christopher M. Bishop.
A free PDF is available online from the author:
Other suggested resources can be found on the Resources Page.
Collaboration Policy
Our ultimate goal is for each student to fully understand the course material. With this goal in mind, we have the following policy:
For quizzes and exams, all work should be done individually, with no collaboration with others whatsoever.
For homeworks and coding practicals, we have the following policy for student work:
You must write anything that will be turned in -- all code and all written solutions -- on your own without help from others. You may not share any code or solutions with others, regardless of if they are enrolled in the class or not.
We do encourage high-level interaction with your classmates. After you have spent at least 10 minutes thinking about the problem on your own, you may verbally discuss assignments with others in the class. You may work out solutions together on whiteboards, laptops, or other media, but you are not allowed to take away any written or electronic information from joint work sessions with others. No notes, no diagrams, and no code. Emails, text messages, and other forms of virtual communication also constitute “notes” and should not be used preparing solutions.
When preparing your solutions, you may always consult textbooks, materials on the course website, or existing content on the web for general background knowledge. However, you cannot ask for answers through any question answering websites such as (but not limited to) Quora, StackOverflow, etc. If you see any material having the same problem and providing a solution, you cannot check or copy the solution provided. If general-purpose material was helpful to you, please cite it in your solution.
Collaboration Statement
Along with all submitted work, you must include the names of any people you worked with, and in what way you worked them (discussed ideas, debugged math, team coding). We may occasionally check in with groups to ascertain that everyone in the group was participating in accordance with this policy.
Academic Integrity Policy
This course will strictly follow the Academic Integrity Policy of Tufts University. Students are expected to finish course work independently when instructed, and to acknowledge all collaborators appropriately when group work is allowed. Submitted work should truthfully represent the time and effort applied.
Please refer to the Academic Integrity Policy at the following URL: https://students.tufts.edu/student-affairs/student-life-policies/academic-integrity-policy
Accessibility
Tufts and the instruction team of COMP 136 strive to create a learning environment that is welcoming students of all backgrounds.
If you feel unwelcome for any reason, please talk to your instructor so we can work to make things better. If you feel uncomfortable talking to members of the teaching staff, consider reaching out to your academic advisor, the department chair, or your dean.
Please see the detailed accessibility policy at the following URL: <https://students.tufts.edu/student-accessibility-services>