Skip to content

🚀 Expert Tier: Modern and 3D Vision

Levels 6–8 · Goal: work at the frontier, and ship to production · ~12–18 weeks at about an hour a day

This tier is being written

Pages marked "Coming soon" list exactly what each chapter will cover. Work through the Beginner and Intermediate tiers first.

You can train and evaluate deep vision models. Now you'll master the ideas behind today's frontier systems (vision transformers, self-supervised and vision-language models, and diffusion), move into video and 3D, and learn to deploy vision models that are fast and reliable.

What you'll be able to do

  • Build a vision transformer from scratch, and explain when ViTs beat CNNs.
  • Use self-supervised features and CLIP-style models for zero-shot classification and image search.
  • Combine foundation models (open-vocabulary detectors and SAM) into working systems.
  • Train a diffusion model, and steer it with guidance.
  • Track multiple objects in video, and work with point clouds and neural rendering.
  • Export, quantize, and deploy vision models for real-time use.

The levels

  • Level 6: Transformers and multimodal

    Vision transformers, self-supervised learning, vision-language models, and open-vocabulary foundation models.

    Start Level 6

  • Level 7: Generative vision

    VAEs and GANs, diffusion models for images, and controllable generation and editing.

    Start Level 7

  • Level 8: Video, 3D, and deployment

    Video understanding, multi-object tracking, point clouds, neural rendering, and deploying vision models.

    Start Level 8

Tier checkpoint

You've mastered this handbook when you can do all of these without notes:

  • Implement patch embedding and a ViT block, and explain the class token and position embeddings.
  • Explain how CLIP is trained, and build a zero-shot classifier with it.
  • Explain the diffusion training objective and classifier-free guidance.
  • Explain how SORT associates detections across frames.
  • Explain the volume rendering equation behind NeRF.
  • Export a model to ONNX, quantize it, and measure the speed-accuracy trade-off.

See the full roadmap for every concept in this tier.