How to Use This Handbook¶
Computer vision is a wide field. It spans signal processing, geometry, statistics, and deep learning, and the people who are genuinely good at it understand why their tools work. This page explains how the handbook is organized and how to study it.
The three tiers¶
| Tier | Levels | Goal | You're ready to move on when… |
|---|---|---|---|
| 🌱 Beginner: Images and Classical Vision | 0–2 | Understand images and master the classical toolkit | You can process images confidently, stitch a panorama, calibrate a camera, and measure depth from stereo |
| 🔧 Intermediate: Deep Learning for Vision | 3–5 | Train and debug CNNs, detectors, and segmentation models | You can take a new dataset to a well-evaluated detector or segmenter |
| 🚀 Expert: Modern and 3D Vision | 6–8 | Work at the frontier, and ship to production | You can build with ViTs, CLIP, and diffusion, handle video and 3D, and deploy efficiently |
Each tier page ends with a checkpoint: what you should be able to do without notes before moving on. The full roadmap lists every concept in every chapter.
Where to start¶
| If you… | Start at |
|---|---|
| Are new to images and vision | Level 0, from the beginning |
| Use OpenCV already, but never learned the theory | Level 1. Try the Level 0 capstone first as a placement test. |
| Know filtering and edges well | Try the Level 1 and Level 2 capstones. If they're easy, start the Intermediate tier. |
| Train CNNs at work | Skim Level 3, then start at Level 4: Object detection. Do the Level 2 geometry chapters at some point: most deep learning practitioners have gaps there. |
If neural networks are new to you
The Intermediate tier assumes you know what a neural network, a loss, and backpropagation are, and the basics of PyTorch. If you don't, work through Levels 6–7 of the companion ML Handbook alongside this handbook's Beginner tier.
How each chapter works¶
- Why it matters: a short story about what goes wrong without this knowledge.
- Concepts: each idea as intuition (with pictures), math (derived, every symbol defined), and code (from scratch first, then the library).
- In practice: complete, runnable code with real output.
- Exercises: 4–6 tasks, with solutions hidden in collapsible boxes.
- Check yourself: questions to answer without notes.
- Key takeaways, further reading, and next.
How to study vision¶
Look at every intermediate result. The single most effective habit in computer vision is visualizing each step: the mask, the edges, the matches, the boxes, the feature maps. Most bugs are obvious the moment you look, and invisible in numbers.
Implement it once from scratch. Writing your own convolution, Harris detector, RANSAC loop, IoU, or NMS once teaches you more than using the library a hundred times. The handbook does this for every core algorithm, and then checks the result against OpenCV or PyTorch.
Test on images you control. Synthetic images with known answers (a square you drew, a warp you applied, a camera you simulated) let you measure exactly how wrong an algorithm is. Real images then tell you how it behaves in the wild.
Do the exercises before opening the solutions, and treat capstones as gates. If you can't finish one without notes, the gap shows you which chapter to revisit.
Keep a "mistakes I made" log. Vision has a famous set of silent bugs: BGR vs RGB, uint8 overflow, (x, y) vs (row, col), off-by-one boxes, and a normalization that differs between training and inference. Write each one down when it bites you.
Conventions¶
| You'll see | It means |
|---|---|
A python block followed by a text block |
The code, then its real output. The copy button copies only the code. |
| A figure after a code block | The real output of that code, generated when the page was built. |
| \(\mathbf{x} = K[R \mid \mathbf{t}]\mathbf{X}\) | Math. The glossary defines the terms. |
=== "From scratch" / === "OpenCV" tabs |
Two implementations of the same thing. |
!!! warning "Common mistake" |
A mistake that practitioners make all the time. |
All randomness is seeded, so you should get the outputs shown, apart from timings and tiny floating-point differences.
What you need¶
- A computer with Python 3.12 or newer. Any OS works.
- No GPU: every example runs on a laptop CPU. The deep learning levels point to free GPU options such as Google Colab where they'd help.
- About an hour a day.
Next: set up your environment.