Introduction to Computer Vision
This course is an introduction to low and intermediate level Computer Vision. We will learn how to design algorithms that process visual scenes to automatically extract information. The course will cover fundamental principles and important applications of computer vision, including image formation, processing, detection and matching features, image segmentation, and multiple views.
Cover photo: “symbolism painting of computer vision” by Linaqruf/anything-v3.0 model
Hours: Monday - Wednesday, 4:30PM – 5:45PM
Location: Joyce Cummings Center, Room 265
Instructor: Dr. Roy Shilkrot roy.shilkrot@tufts.edu roy@cs.tufts.edu LinkedIn GitHub Twitter
Office hours: Schedule online with Prof. Roy: https://calendly.com/royshilkrot/15min
Prerequisites: Algorithms and data structures, linear algebra (I or II), statistics and probability theory (I), calculus (I), working knowledge of Python
Teaching assistant: Li Yukun Yukun.Li@tufts.edu
TA Office hours: Wednesday 6:30 pm-7:30 pm: https://tufts.zoom.us/j/3088684222
Piazza: https://piazza.com/tufts/spring2023/cs15009
By the end of the semester, students should be able to:
We will follow chapters from the books below:
|
Class |
Date |
Content |
Assignment |
|
|
1 |
1/18 |
Introduction: Class, Computer Vision, Math |
||
|
2 |
1/23 |
Image Formation: Human Vision, Optics, Physics, Cameras |
||
|
3 |
1/25 |
Image Formation cont.: Perspective projection, Pinhole camera model, Digital Imaging, Colorspaces, Sampling |
HW1: Hello Vision World |
|
|
4 |
1/30 |
Image Processing: Filters, Edges: Convolutions, Separability, Local operators, Frequency domain and Fourier Transform, Histograms, non-linear (median, bilateral), Pyramids |
||
|
5 |
2/1 |
Features: Interests points, Corners, Harris, Blobs, FAST, Multiscale |
HW2: Pyramid blending, Corners & Blobs, Key points |
|
|
6 |
2/6 |
Features: Descriptors, Gradient Histograms, Edges & Contours, Snakes & Scissors |
||
|
7 |
2/8 |
Geometric / Planar Transforms Parameterization, Feature Matching, Least Squares optimization |
HW3: Feature Extraction & Matching, Snakes |
|
|
8 |
2/13 |
Image Alignment, RANSAC, Stitching, Panoramas, Cylindrical coordinates, Global optimization / Bundle Adjustment, Blending |
||
|
9 |
2/15 |
Tracking: Motion estimation parameterizations, CONDENSATION, Particle and Kalman Filtering |
HW4: Panorama Stitching |
|
|
2/20 |
No class - President’s day |
|||
|
10 |
2/22 |
Tracking: Correlation Filters, Optical flow, Lucas-Kanade, Shi-Tomasi |
HW5: Tracking |
|
|
11 |
2/27 |
Multiview 1: Intro, Camera Calibration, Calibration and Projection Matrices |
||
|
12 |
3/1 |
Multiview 2: Parallel Motion Stereo, Depth and Disparity, Epipolar Geometry, Essential and Fundamental Matrices |
Mid-term prep |
|
|
13 |
3/6 |
Mid-term prep |
Mid-term prep |
|
|
14 |
3/8 |
Mid-term (in class) |
||
|
15 |
3/13 |
Multiview 3: Triangulation, Camera Pose Estimation, Structured Light |
||
|
16 |
3/15 |
Bundle Adjustment, Intro to Visual Machine Learning |
HW6: MVG, Bundle Adjustment |
|
|
3/20 |
No class - spring break |
|||
|
3/22 |
||||
|
17 |
3/27 |
Linear Models, Linear Regression and Classification, Logistic Regression |
||
|
18 |
3/29 |
Linear SVMs, Neural Networks Online: Gradient descent, Backpropagation and Learning Practices |
||
|
19 |
4/3 |
Convolutional Neural Networks, Deep Learning |
HW7: Hello CNNs, Classification |
|
|
20 |
4/5 |
Object detection 1: V-J, Region Proposals Nets, R-CNN, SPP, Fast-RCNN |
||
|
21 |
4/10 |
Object detection 2: Faster-RCNN, YOLOs, SSD Segmentation 1: Gestalt theory, Clustering, Meanshift, GMM, watershed, superpixels, Graph-Cut |
||
|
22 |
4/12 |
Segmentation 2: Types of Segmentation, FCNs, U-net, Upconvolutions, SegNet |
HW8: Object Detection |
|
|
4/17 |
No class - Patriot’s day |
|||
|
23 |
4/21 |
Vision and Language: Introduction, VQA, Captioning, RNNs and Language Models, LSTMs |
||
|
24 |
4/24 |
Vision and Language II: Attention and Visual Attention, Word Embeddings, VQA, Transformers, ViT |
||
|
25 |
4/26 |
Vision and Language III: CLIP, Generative modeling, AE, VAE, GANs, Vision Applications, CLIP-guided Diffusion and Text-to-Image |
||
|
26 |
5/1 |
Final class. What have we missed? Further reading, final exam prep |
Final exam prep |
|
Problem sets: Nine (9) problem sets will be given during the class. Any material regarding the set solution will be submitted electronically. Homework will be due up to one or two weeks from the assigned date at midnight. Late assignments are penalized at 20% for each 24 hours’ delay. No homework will be accepted one week after the deadline. If a serious illness or another major life event prevents you from completing a homework assignment on time, you should report the event to your instructions team, after which alternate arrangements can be made. Reporting must be done before the assignment in question is due.
Tests: There will be two (2) tests over the course of the semester. The tests will be a combination of multiple choices and open-ended answers. Books must be closed and no electronic devices are allowed. If, for any reason you must miss a test, you must inform me before the day of the exam and schedule a make-up session. Please Note: If you wish to dispute a grade, it is mandatory that you do so within one week of receiving the grade. After such a term, the grade will be considered final. Note: Grading is generous, re-grading is strict.
Final grades: You must show proficiency in all grading areas to pass the class. A failing average (below 50%) in any of the grading areas (problem sets, tests, attendance) will result in a failing grade in the class.
Your final grade will be determined using the following percentage breakdown:
|
Problem Sets |
50% |
|
Tests |
40% (mid 13%, final 27%) |
|
Attendance, Behavior and Attitude |
10% |
Piazza will be our primary means of communication. All course announcements will also be made through Piazza.
Rather than emailing questions to the teaching staff, we encourage you to post your questions as a public discussion. You are also encouraged to help each other, as long as the question does not contain any code or portion of a problem set answer. In such a case, the question must be made private (please, refer to the section Academic Honesty below).
To schedule office hours outside the posted times, please email any of the teaching staff or post a private message on Piazza.
Science is, to its core, a collaborative effort. The advantages of coming together for examination or comparison, sometimes even just to explain the problem, are well known. I strongly encourage students to discuss course material, problems, and applications outside the classroom with the teaching staff and other students. You are also encouraged to form study groups for the tests.
Having said that, integrity and honesty are equally important qualities of any future academic, scientist or engineer. We take plagiarism very seriously. You must do homework, the problem sets and the final projects on your own. If you need help, the teaching staff will be more than happy to help you!
Academic Integrity Policy: Tufts holds its students strictly accountable for adherence to academic integrity. The consequences for violations can be severe. It is critical that you understand the requirements of ethical behavior and academic work as described in Tufts’ Academic Integrity handbook. If you ever have a question about the expectations concerning a particular assignment or project in this course, be sure to ask me for clarification. The Faculty of the School of Arts and Sciences and the School of Engineering are required to report suspected cases of academic integrity violations to the Dean of Student Affairs Office. If I suspect that you have cheated or plagiarized in this class, I must report the situation to the dean.
