Computational imaging is a rapidly growing field that lies at the intersection of optics, signal processing, computer vision, and machine learning. It enables imaging systems that go beyond the limits of traditional cameras, and is foundational to technologies ranging from scientific instruments to everyday consumer devices like smartphones and AR headsets. In this seminar-style course, we will study core ideas and breakthrough papers across a wide range of topics, including event-based vision, single-photon imaging, light field cameras, time-of- flight cameras, and imaging hidden scenes. The course will cover both hardware innovations—such as novel sensors that enable single-shot 3D imaging—and computational techniques that can, for example, recover sound from video. We will also examine how physics-inspired neural representa- tions and differentiable models are reshaping classical problems in imaging, from denoising to 3D perception in complex environments.
Topics include:
Lectures: Tuesdays 12:00-1:15pm and Thursdays 12:00-1:15pm in Lawson Computer Science B134.
Instructor office hours: Tuesdays 2:00-3:00pm (Lawson Computer Science 3154N) starting on September 2; discussion about projects, lecture material, etc. Office hours can also be on Zoom if requested.Helpful background:
Textbook: While there is no required textbook for the course, the following materials may support a deeper understanding::
Contact: Course announcements and general information will be posted on the course forum on BrightSpace.
The course project consists of a research project with both a written report and a source code submission. Projects may be completed individually or in teams of up to two students. Submit one proposal and one final report per team. The expected workload should be proportional to team size.
Project proposal is due on Friday October 25 (5% of total grade). Final report is due at the end of classes (40% of total grade).
The final report grade takes into account your source code submission (code organization and documentation) and the report itself (appropriate format and length, abstract, introduction, related work, description of your method, quantitative and qualitative evaluation of your method, results, discussion & conclusion, bibliography).
You can work in teams of up to 2 students for the project. Submit only one proposal and final report for each team. The expected amount of work is relative to the number of team members, so if two teams work on a similar project, we'd expect less work from a smaller team.
The project proposal is a 1-2 page document that should contain the following elements: clear motivation of your idea, a discussion of related work along at least 3 scientific references (i.e., scientific papers not blog articles or websites), an overview of what exactly your project is about and what the final goals are, milestones for your team with a timeline and intermediate goals. After proposal submission, revisions may be suggested.
The final project report should look like a short (~6 pages) conference paper. We expect the following sections, which are standard practice for conference papers: abstract, introduction, related work, theory (i.e., your approach), analysis and evaluation, results, discussion and conclusion, references. Use the CVPR 2025 LaTeX template for your report. A detailed rubric can be found on BrightSpace under the final project report assignment.
All homework is due at midnight on the due date. For role-specific homework, there will be a 30% deduction if you submit late, but before the start of that week's lecture (i.e., if you submit anytime between 12:01am and 12pm on Tuesday or Thursday). No homework will be accepted after the start of lecture.
If you need more time to submit your project proposal or final project report, you will need to discuss your timeline with the instructor and get approval at least 1 week before the posted due date.
You may use any tools you find productive in preparing your reports. But you are responsible for any misrepresentations, inaccuracies, or plagiarism. If we find such defects in submitted work, we reserve the right to exact penalties, which may range from grade deductions to reporting academic misconduct depending on the severity of the offense.
Week | Date | Description | Material | Readings | Event | Deadline |
---|---|---|---|---|---|---|
Tue Aug 26 |
Lecture 1: Course intro |
[slides]
|
Marc Levoy's course on digital photography | |||
Thu Aug 28 |
Lecture2: Digital photography | [slides] | ||||
Tue |
Lecture 3: Pinholes and lenses |
[slides] | Marc Levoy's course on digital photography | |||
Thu |
Lecture 4: Selected topics | |||||
Imaging and Sound | ||||||
Tue |
The visual microphone: Passive recovery of sound from video | [Paper] | ||||
Thu |
Dual-Shutter Optical Vibration Sensing | [Paper] | ||||
Optical Coding | ||||||
Tue |
Light field photography with a hand-held plenoptic camera | [Paper] | ||||
Thu |
DiffuserCam: lensless single-exposure 3D imaging | [Paper] | ||||
Computational Light Transport | ||||||
Tue |
Femto-photography: capturing and visualizing the propagation of light | [Paper] | ||||
Thu |
Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging | [Paper] | ||||
Single-Photon Imaging | ||||||
Tue |
Passive inter-photon imaging | [Paper] | ||||
Thu |
Quanta Burst Photography | [Paper] | ||||
Single-Photon 3D Sensing | ||||||
Tue |
First-Photon Imaging | [Paper] | ||||
Thu |
Opportunistic Single-Photon Time of Flight | [Paper] | ||||
Tue |
Fall break | |||||
Thu |
Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera | [Paper] | ||||
Tue |
Project proposal presentation |
|||||
Thu |
Projects Q&A (on Zoom) | |||||
Fri |
Project proposals due at 11:59pm | |||||
Neural Representations & Computational Imaging I | ||||||
Tue |
Flying with Photons | [Paper] | ||||
Thu |
Neural Inverse Rendering from Propagating Light | [Paper] | ||||
Neural Representations & Computational Imaging II | ||||||
Tue |
NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images | [Paper] | ||||
Thu |
Learning Lens Blur Fields | [Paper] | ||||
TBD | ||||||
Tue |
Minimalist Camera/flutter shutter | [Paper] | ||||
Thu |
Invited Talk | |||||
Deep Optics | ||||||
Tue |
Deep Optics for Single-shot HDR Imaging | [Paper] | ||||
Thu |
Curriculum learning for ab initio deep learned refractive optics | [Paper] | ||||
Generative Computational Imaging | ||||||
Tue |
Generative Photography | [Paper] | ||||
Thu |
Thanksgiving break | |||||
Tue |
Invited talk:TBD |
|
||||
Thu |
Project Q&A and debugging | |||||
Tue |
Invited Talk:TBD |
|||||
Thu |
Project Q&A | |||||
Sat |
Project reports due at 11:59pm |
The course is adapted from CSC2529 at the University of Toronto by David Lindell. Some of the materials used in class build on that from other instructors, including Yannis Gkioulekas, Marc Levoy, Fredo Durand, Ramesh Raskar, Kyros Kutulakos, Shree Nayar, Paul Debevec, Matthew O'Toole and others, as noted in the slides. Feel free to use these slides for academic or research purposes, but please maintain all acknowledgments. This webpage is based on the website for CSC2529 at the University of Toronto.