Introduction to Computer Vision
CS150-04 | Fall 2023
This course is an introduction to low and intermediate level of classic and modern Computer Vision. We will learn how to design algorithms that process visual scenes to automatically extract information. The course will cover fundamental principles and important applications of computer vision, including image formation, processing, detection and matching features, image segmentation, and multiple views. We will also cover the basics of machine learning and deep learning for computer vision. The course will include a mix of lectures, paper readings, and hands-on Python programming assignments.
Cover photo: "introduction to computer vision, a digital painting” by SDv1.5
Course Information
Hours: Tuesday & Thursday, 4:30PM – 5:45PM
Location: Aidekman Center, Room 012
Instructor: Dr. Roy Shilkrot roy.shilkrot@tufts.edu | roy@cs.tufts.edu | LinkedIn | GitHub | Twitter/X | YouTube
Office hours: Schedule online with Prof. Roy: https://calendly.com/royshilkrot/15min
Prerequisites: Algorithms and data structures, linear algebra (I or II), statistics and probability theory (I), calculus (I), working knowledge of Python
Teaching assistant: Yukun Li Yukun.Li@tufts.edu
TA Office hours: TBD
Piazza: https://piazza.com/tufts/fall2023/cs15004
Course Goals
By the end of the semester, students should be able to:
- Identify and understand the fundamental aspects and techniques of Computer Vision
- Interpret and analyze a given problem using these techniques
- Adopt and Implement the appropriate techniques to solve a given problem
Textbooks
We will follow chapters from the books below:
- Computer Vision: Algorithms and applications, 2nd Ed (2022). Richard Szeliski. The book PDF is freely available online (http://szeliski.org/Book)
- Simon J.D Prince. Computer Vision Models (2012)
- Aeorlien Geron. Hands on Machine Learning 3rd ed. (2022)
- Francois Chollet. Deep Learning with Python 2nd ed. (2021)
- Excerpts from Hartley & Zisserman (2004), and Marr (2010)
As well as reference videos from Prof. Shree Nayar's lecture series First Principles of Computer Vision (https://www.youtube.com/@firstprinciplesofcomputerv3258)
Syllabus
| Class | Content | Links | Assignment |
|---|---|---|---|
| 1 | Introduction: Class, Computer Vision, Math | slides video | |
| 2 | Image Formation: Human Vision, Optics, Physics, Cameras | slides video | |
| 3 | Image Formation cont.: Perspective projection, Pinhole camera model, Digital Imaging, Colorspaces | slides video | HW1: Hello Vision World |
| 4 | Image Processing: Sampling, Filters, Edges: Convolutions, Separability, Local operators, Frequency domain and Fourier Transform, Histograms, non-linear (median, bilateral), Pyramids | slides video | |
| 5 | Features: Interests points, Corners, Harris, Blobs, FAST, Multiscale | slides video | HW2: Pyramid blending, Corners & Blobs, Key points |
| 6 | Features: Descriptors, Gradient Histograms, Edges & Contours, Snakes & Scissors | slides video | |
| 7 | Geometric / Planar Transforms Parameterization, Feature Matching, Least Squares optimization | slides video | |
| 8 | Image Alignment, RANSAC, Stitching, Panoramas, Cylindrical coordinates, Global optimization / Bundle Adjustment, Blending | slides video | HW3: Feature Extraction & Matching, Snakes |
| 9 | Tracking: Meanshift, Motion estimation parameterizations | slides video | HW4: Panorama Stitching |
| 10a | Tracking: CONDENSATION, Particle Filtering, Correlation Filters | slides video | |
| 10b | Tracking: Optical flow, Lucas-Kanade, Shi-Tomasi | slides video | HW5: Tracking |
| 11 | Multiview 1: Intro, Camera Calibration, Calibration and Projection Matrices | slides video | |
| 12 | Multiview 2: Parallel Motion Stereo, Depth and Disparity, Epipolar Geometry, Essential and Fundamental Matrices | slides video | |
| 13 | Mid-term prep | Slides Video | |
| 14 | Mid-term (in class) | ||
| 15 | Multiview 3: Triangulation, Camera Pose Estimation, Structured Light | slides video | |
| 16 | Bundle Adjustment, Intro to Visual Machine Learning | slides video | HW6: MVG, Bundle Adjustment |
| 17 | Linear Models, Linear Regression and Classification, Logistic Regression | Slides Video | |
| 18 | Linear SVMs, Neural Networks. Online: Gradient descent, Backpropagation and Learning Practices | Video | |
| 19 | Convolutional Neural Networks, Deep Learning | Slides Video | HW7: Hello CNNs, Classification |
| 20 | Object detection 1: V-J, Region Proposals Nets, R-CNN, SPP, Fast-RCNN, YOLO, SSD | Slides Video | |
| 21 | Vision and Language: Introduction, VQA, Captioning, RNNs and Language Models, LSTMs | HW8: Object Detection | |
| 22 | Vision and Language II: Attention and Visual Attention, Word Embeddings, VQA, Transformers, ViT | ||
| 23 | Final class. What have we missed? Further reading, final exam prep | ||
| Final exam (timing TBD) |
Problem Sets, Tests
Problem sets: Nine (9) problem sets will be given during the class. Any material regarding the set solution will be submitted electronically. Homework will be due up to one or two weeks from the assigned date at midnight. Late assignments are penalized at 20% for each 24 hours’ delay. No homework will be accepted one week after the deadline. If a serious illness or another major life event prevents you from completing a homework assignment on time, you should report the event to your instructions team, after which alternate arrangements can be made. Reporting must be done before the assignment in question is due.
Tests: There will be two (2) tests over the course of the semester. The tests will be a combination of multiple choices and open-ended answers. Books must be closed and no electronic devices are allowed. If, for any reason you must miss a test, you must inform me before the day of the exam and schedule a make-up session. Please Note: If you wish to dispute a grade, it is mandatory that you do so within one week of receiving the grade. After such a term, the grade will be considered final. Note: Grading is generous, re-grading is strict.
Final grades: You must show proficiency in all grading areas to pass the class. A failing average (below 50%) in any of the grading areas (problem sets, tests, attendance) will result in a failing grade in the class. Your final grade will be determined using the following percentage breakdown:
| Grading Area | Percentage |
|---|---|
| Problem Sets | 50% |
| Tests | 40% (mid 13%, final 27%) |
| Attendance, Behavior, Attitude, Decorum | 10% |
Communications
Piazza will be our primary means of communication. All course announcements will also be made through Piazza. Rather than emailing questions to the teaching staff, we encourage you to post your questions as a public discussion. You are also encouraged to help each other, as long as the question does not contain any code or portion of a problem set answer. In such a case, the question must be made private (please, refer to the section Academic Honesty below). To schedule office hours outside the posted times, please email any of the teaching staff or post a private message on Piazza.
Academic Honesty
Science is, to its core, a collaborative effort. The advantages of coming together for examination or comparison, sometimes even just to explain the problem, are well known. I strongly encourage students to discuss course material, problems, and applications outside the classroom with the teaching staff and other students. You are also encouraged to form study groups for the tests.
Having said that, integrity and honesty are equally important qualities of any future academic, scientist or engineer. We take plagiarism very seriously. You must do homework, the problem sets and the final projects on your own. If you need help, the teaching staff will be more than happy to help you!
Using AI. We encourage and expect you to use AI tools such as GitHub Copilot and ChatGPT to aid you in this class, as would be expected of you by your future employers or colleagues. However, understand the risks of using these tools without a solid understanding of their inner workings. Research hallucinations and AI plagiarism before using any products of AI tools as submissions in this class. OpenAI has helpful information on the topic, as well as the faculty in Wharton.
Academic Integrity Policy
Tufts holds its students strictly accountable for adherence to academic integrity. The consequences for violations can be severe. It is critical that you understand the requirements of ethical behavior and academic work as described in Tufts’ Academic Integrity handbook. If you ever have a question about the expectations concerning a particular assignment or project in this course, be sure to ask me for clarification. The Faculty of the School of Arts and Sciences and the School of Engineering are required to report suspected cases of academic integrity violations to the Dean of Student Affairs Office. If I suspect that you have cheated or plagiarized in this class, I must report the situation to the dean.