## Computational Imaging

### Instructor Requests

A comprehensive and up-to-date textbook and reference for computational imaging, which combines vision, graphics, signal processing, and optics.

Computational imaging involves the joint design of imaging hardware and computer algorithms to create novel imaging systems with unprecedented capabilities. In recent years such capabilities include cameras that operate at a trillion frames per second, microscopes that can see small viruses long thought to be optically irresolvable, and telescopes that capture images of black holes. This text offers a comprehensive and up-to-date introduction to this rapidly growing field, a convergence of vision, graphics, signal processing, and optics. It can be used as an instructional resource for computer imaging courses and as a reference for professionals. It covers the fundamentals of the field, current research and applications, and light transport techniques.

The text first presents an imaging toolkit, including optics, image sensors, and illumination, and a computational toolkit, introducing modeling, mathematical tools, model-based inversion, data-driven inversion techniques, and hybrid inversion techniques. It then examines different modalities of light, focusing on the plenoptic function, which describes degrees of freedom of a light ray. Finally, the text outlines light transport techniques, describing imaging systems that obtain micron-scale 3D shape or optimize for noise-free imaging, optical computing, and non-line-of-sight imaging. Throughout, it discusses the use of computational imaging methods in a range of application areas, including smart phone photography, autonomous driving, and medical imaging. End-of-chapter exercises help put the material in context.

Expand/Collapse All
Contents (pg. v)
List of Figures (pg. xi)
List of Tables (pg. xxix)
Preface (pg. xxxi)
1. Introduction to Computational Imaging (pg. 1)
1.1 What Is Computational Imaging? (pg. 1)
1.2 Historical Roots of Computational Imaging (pg. 3)
1.3 Modern Uses of Computational Imaging (pg. 4)
1.4 Roadmap of the Book (pg. 6)
Part I: Toolkit (pg. 9)
2. Imaging Toolkit (pg. 11)
2.1 Optics (pg. 11)
2.2 Image Sensors (pg. 35)
2.3 Illumination (pg. 48)
3. Computational Toolkit (pg. 67)
3.1 Modeling: Forward vs. Inverse Problems (pg. 67)
3.2 Mathematical Tools (pg. 68)
3.3 Model-Based Inversion (pg. 85)
3.4 Data-Driven Inversion Techniques (pg. 99)
3.5 Hybrid Inversion Techniques (Data Driven and Model Based) (pg. 115)
Part II: Plenoptic Imaging (pg. 125)
4. Spatially Coded Imaging (pg. 127)
4.1 Coding the Aperture (pg. 128)
4.2 Coding the Sensor (pg. 138)
4.3 Coding the Illumination (pg. 151)
4.4 Further Research (pg. 155)
5. Temporally Coded Imaging (pg. 169)
5.1 A Brief History of the Time-of-Flight Revolution (pg. 170)
5.2 Optical Time-Resolved Imaging (pg. 173)
5.3 Time-Resolved Image Formation Model (pg. 176)
5.4 Lockin Sensorâ€“based 3D Imaging (pg. 181)
5.5 Application Areas (pg. 185)
5.6 Summary of Recent Advances and Further Applications (pg. 196)
5.7 Related Optical Imaging Techniques (pg. 201)
6. Light Field Imaging and Display (pg. 211)
6.1 Historical Highlight: Lippmann Light Field Camera (1908) (pg. 212)
6.2 Light Field Processing (pg. 212)
6.3 Light Field Capture (pg. 225)
6.4 Light Field Displays (pg. 238)
7. Polarimetric Imaging (pg. 253)
7.1 Principles of Polarization (pg. 253)
7.2 Full Stokes Imaging (pg. 260)
7.3 3D Shape Reconstruction (pg. 263)
7.4 Imaging through Scattering Media (pg. 267)
7.5 Reflectance Decomposition Using Polarimetric Cues (pg. 276)
8. Spectral Imaging (pg. 287)
8.1 Spectral Effects on Light-Matter Interaction (pg. 287)
8.2 Color Theory (pg. 293)
8.3 Optical Setups for Spectral Imaging (pg. 297)
8.4 Computational Methods for Analyzing Spectral Data (pg. 304)
Part III: Shading and Transport of Light (pg. 315)
9. Programmable Illumination and Shading (pg. 317)
9.1 Scene Reflectance and Photometry (pg. 317)
9.2 Shape from Intensity (pg. 320)
9.3 Multiplexed Illumination (pg. 336)
9.4 Applications in Graphics (pg. 338)
10. Light Transport (pg. 357)
10.1 Motivation (pg. 357)
10.2 Light Transport Matrix (pg. 359)
10.3 Relaxations of Inverse Light Transport (pg. 363)
10.4 Non-Line-of-Sight Imaging (pg. 378)
10.5 Applications (pg. 401)
Glossary (pg. 415)
References (pg. 421)
Index (pg. 443)

#### Ayush Bhandari

Ayush Bhandari is Assistant Professor of Electrical and Electronic Engineering at Imperial College London.

Achuta Kadambi is Assistant Professor of Electrical Engineering and Computer Science at the University of California, Los Angeles.