The Computer Vision Handbook

From pixels and filters to detectors, vision transformers, and 3D

A hands-on handbook that takes you from "what is an image?" to a deep understanding of modern computer vision. You learn every idea three ways: the intuition (with pictures, because vision is visual), the math, and the code. You implement the core algorithms from scratch before using OpenCV and PyTorch, and every level ends with a capstone project.

Start here See the full roadmap Set up your environment

Three tiers, nine levels

The handbook has 44 chapters in nine levels, grouped into three tiers.

Status

The handbook is being written one tier at a time. Pages marked "Coming soon" list exactly what each chapter will cover.

• 🌱 Beginner: Images and Classical Vision

  ---

  Levels 0–2. What images really are, image processing from first principles, and classical vision: features, panoramas, camera calibration, stereo depth, and tracking.

  • Level 0: Images and tools
  • Level 1: Image processing
  • Level 2: Classical vision

  Beginner tier

• 🔧 Intermediate: Deep Learning for Vision

  ---

  Levels 3–5. CNNs from the inside, object detection from IoU to DETR, and segmentation, pose, depth, and promptable models like SAM.

  • Level 3: CNNs for vision
  • Level 4: Object detection
  • Level 5: Segmentation and dense prediction

  Intermediate tier

• 🚀 Expert: Modern and 3D Vision

  ---

  Levels 6–8. Vision transformers, self-supervised and vision-language models, diffusion, video, 3D and neural rendering, and deployment.

  • Level 6: Transformers and multimodal
  • Level 7: Generative vision
  • Level 8: Video, 3D, and deployment

  Expert tier

The learning path

flowchart LR
    subgraph B["🌱 Beginner"]
        L0["Images<br/>and tools"] --> L1["Image<br/>processing"] --> L2["Classical<br/>vision"]
    end
    subgraph I["🔧 Intermediate"]
        L3["CNNs"] --> L4["Object<br/>detection"] --> L5["Segmentation<br/>and dense"]
    end
    subgraph E["🚀 Expert"]
        L6["Transformers<br/>and multimodal"] --> L7["Generative<br/>vision"] --> L8["Video, 3D,<br/>deployment"]
    end
    L2 -- "tier checkpoint" --> L3
    L5 -- "tier checkpoint" --> L6

Why start with classical vision?

It's tempting to jump straight to deep learning. But the classical toolkit is everywhere in modern systems: every detector's data pipeline uses image processing, every AR app and self-driving stack uses camera geometry, and every 3D reconstruction uses features, RANSAC, and triangulation. Understanding convolution as a hand-designed filter also makes it obvious what a CNN is learning. The Beginner tier gives you that foundation. If you already have it, the start-here guide explains how to skip ahead.

How each chapter is organized

Part	What you get
1. Why it matters	A short real story where this knowledge saved a project, or its absence sank one.
2. Concepts	Intuition → math → code, with before/after images, diagrams, and derivations.
3. In practice	Complete, runnable code with real output, and the common mistakes (BGR vs RGB, overflow, box formats…).
4. Exercises	4–6 tasks, from easy to hard, each with a hidden solution.
5. Check yourself	Questions to answer without notes.

Companion handbook

General machine learning and deep learning foundations (linear algebra, probability, classical ML, backpropagation, and PyTorch) live in the companion ML Handbook. This handbook links to it where you need a refresher, instead of repeating it.

Ready?

Start here Cheat sheets