Glossary

Every important term in the handbook, defined briefly. Chapters define each term properly the first time it appears. Use this page as a quick reminder. For general ML and deep learning terms (loss, gradient, overfitting, and so on), see the ML Handbook glossary.

A · B · C · D · E · F · G · H · I · K · L · M · N · O · P · Q · R · S · T · U · V · W

A

Affine transformation

A transformation that preserves straight lines and parallelism: any combination of translation, rotation, scaling, and shear. It's a 2×3 matrix in 2D.

Aliasing

Artifacts (jagged edges, moiré patterns) that appear when an image is sampled too coarsely for the detail it contains. You prevent it by blurring before downsampling.

Anchor box

A predefined box of a fixed size and aspect ratio, placed at every location. Many detectors predict offsets from anchors rather than boxes from scratch.

Aperture problem

Through a small window, you can only measure motion perpendicular to an edge, not along it. That's why optical flow needs extra assumptions.

Augmentation

Randomly transforming training images (flips, crops, color jitter, and so on) to make a model robust and reduce overfitting.

Average precision (AP)

The area under a precision-recall curve for one class. mAP averages it over classes, and in COCO also over IoU thresholds from 0.5 to 0.95.

B

Backbone

The feature-extraction part of a vision model (such as a ResNet or ViT) that task-specific heads build on.

Baseline (stereo)

The distance between two cameras' centers. Depth is proportional to focal length × baseline ÷ disparity.

Bayer filter

The mosaic of red, green, and blue color filters over a camera sensor. Each pixel measures one color; demosaicing reconstructs full color.

BGR

OpenCV's default channel order (blue, green, red). Most other libraries use RGB. Mixing them up swaps red and blue.

Bilinear interpolation

Estimating a pixel value at a non-integer position as a weighted average of its four nearest neighbors.

Bit depth

The number of bits per channel value: 8 bits gives 256 levels, and 16 bits gives 65,536.

Bounding box

A rectangle around an object, in formats such as (x_min, y_min, x_max, y_max), (x, y, w, h), or normalized (center x, center y, w, h).

Bundle adjustment

Jointly refining camera poses and 3D points to minimize reprojection error across all views.

C

Calibration (camera)

Estimating a camera's intrinsic parameters (and distortion), usually from images of a known pattern such as a checkerboard.

Canny edge detector

A classic edge detector: Gaussian smoothing, gradients, non-maximum suppression, then hysteresis thresholding.

CHW / HWC

Channel-first (PyTorch) vs channel-last (NumPy, OpenCV, Pillow) memory layouts for image tensors.

CLAHE

Contrast-limited adaptive histogram equalization. It equalizes contrast in local tiles while limiting noise amplification.

Class token

An extra learned token prepended to a ViT's patch sequence. Its final representation is used for classification.

CLIP

A model trained contrastively on image-text pairs, so images and captions share an embedding space. It enables zero-shot classification and search.

Color space

A way of representing color as numbers: RGB, HSV (hue, saturation, value), Lab (perceptual), or YCbCr (luma plus chroma).

Connected components

Groups of touching foreground pixels in a binary image, each given its own label.

Contour

The boundary curve of a region, usually stored as a list of points.

Convolution

Sliding a flipped kernel over an image and computing weighted sums. In deep learning, "convolution" usually means correlation, which doesn't flip the kernel.

Correspondence

A pair of points in two images that are projections of the same 3D point.

D

DDPM / DDIM

Denoising diffusion probabilistic models, and a faster deterministic sampler for them.

Demosaicing

Reconstructing a full-color image from a Bayer-filtered sensor reading.

Depthwise-separable convolution

A convolution split into a per-channel spatial convolution plus a 1×1 channel-mixing convolution. It's far cheaper than a standard convolution.

DETR

A detection transformer that predicts a set of boxes directly, using bipartite matching in training and no NMS.

Dice coefficient

An overlap measure for segmentation: 2|A ∩ B| / (|A| + |B|). It's closely related to IoU.

Diffusion model

A generative model trained to reverse a gradual noising process. It generates images by iteratively denoising random noise.

Dilation (morphology)

Growing the foreground of a binary image with a structuring element. In CNNs, a dilated convolution spaces out kernel taps to enlarge the receptive field.

Disparity

The horizontal shift of a point between rectified stereo images. It's inversely proportional to depth.

Distortion (lens)

Deviation from the ideal pinhole projection. Radial distortion makes straight lines curve (barrel or pincushion); tangential distortion comes from lens misalignment.

Distance transform

An image where each pixel's value is its distance to the nearest background pixel.

E

Epipolar line

The line in one image on which the match of a point in the other image must lie.

Epipole

The point where the line joining two camera centers meets an image plane.

Erosion

Shrinking the foreground of a binary image with a structuring element.

Essential matrix

A 3×3 matrix relating corresponding normalized image points of two calibrated cameras: \(\mathbf{x}_2^\top E\,\mathbf{x}_1 = 0\).

Extrinsic parameters

A camera's rotation and translation relative to the world.

F

Feature descriptor

A vector describing the neighborhood of a keypoint (SIFT: 128 floats; ORB: 256 bits), used for matching.

Feature map

The output of a convolutional layer: one 2D map per channel.

Feature pyramid network (FPN)

A top-down architecture with lateral connections that gives semantically strong features at every scale.

FID

Fréchet Inception Distance. It compares feature statistics of generated and real images; lower is better.

Fine-tuning

Continuing to train a pretrained model on a new task or dataset.

Focal length

The distance from the camera center to the image plane, usually expressed in pixels in the intrinsic matrix.

Focal loss

A loss that down-weights easy examples to handle extreme foreground-background imbalance in detection.

Fourier transform

Represents an image as a sum of sinusoids of different frequencies, orientations, and phases.

Fundamental matrix

A 3×3 rank-2 matrix relating corresponding pixel coordinates of two uncalibrated views: \(\mathbf{x}_2^\top F\,\mathbf{x}_1 = 0\).

G

Gamma correction

A nonlinear brightness mapping, \(V_{out} = V_{in}^{\gamma}\). sRGB images are gamma-encoded, so pixel values aren't proportional to light intensity.

GAN

A generative adversarial network: a generator and a discriminator trained against each other.

Gaussian blur

Smoothing with a Gaussian kernel. It's separable and the standard way to reduce noise before differentiation.

Gaussian splatting

Representing a 3D scene as many 3D Gaussians that can be rasterized very fast for novel view synthesis.

Grad-CAM

A visualization that highlights the image regions most responsible for a CNN's prediction, using gradients of the class score with respect to a feature map.

H

Harris corner detector

Finds corners where image intensity changes strongly in all directions, using the eigenvalues of the structure tensor.

Histogram equalization

Remapping intensities through their cumulative distribution so the output histogram is roughly flat, which increases contrast.

HOG

Histogram of Oriented Gradients: a hand-crafted descriptor built from local gradient-orientation histograms. It's the classic pedestrian-detection feature.

Homogeneous coordinates

Representing a 2D point as (x, y, 1), up to scale, so translations and projections become matrix multiplications.

Homography

A 3×3 projective transformation between two planes, or between two images taken by a purely rotating camera.

Hough transform

Detecting lines or circles by voting in parameter space.

Hungarian algorithm

Solves the optimal one-to-one assignment problem. It's used in DETR training and in tracking.

I

Instance segmentation

Labeling each pixel with an object instance, so two touching people get different masks.

Integral image

An image where each pixel holds the sum of all pixels above and to its left, making rectangle sums constant time. Used by Viola-Jones.

Interpolation

Estimating values between known pixels: nearest neighbor, bilinear, bicubic, or Lanczos.

Intrinsic matrix (K)

The 3×3 matrix of a camera's focal lengths, principal point, and skew.

IoU (intersection over union)

The area of overlap divided by the area of union, for two boxes or masks. It's the standard overlap measure in detection and segmentation.

ICP (iterative closest point)

Aligns two point clouds by alternating nearest-neighbor matching and rigid transform estimation.

K

Kalman filter

A recursive estimator that predicts a state (such as position and velocity) and corrects it with noisy measurements.

Kernel (filter)

A small matrix of weights used in convolution.

Keypoint

A distinctive, repeatable image location, such as a corner or blob, used for matching. In pose estimation, it's a body joint.

L

Laplacian

The sum of second derivatives. It responds to rapid intensity changes and is used for edge and blob detection, and for measuring sharpness.

Laplacian pyramid

A multiscale representation storing the detail lost between successive levels of a Gaussian pyramid. It allows exact reconstruction and seamless blending.

Latent diffusion

Running diffusion in a compressed latent space (from an autoencoder) instead of in pixel space. Used by Stable Diffusion.

LiDAR

A sensor that measures distance by timing laser pulses, producing point clouds.

M

Mask

A per-pixel image (binary or labeled) marking which pixels belong to an object or class.

Morphology

Shape-based operations on images (erosion, dilation, opening, closing) using a structuring element.

Multi-object tracking (MOT)

Following many objects across video frames while keeping their identities.

N

NeRF

A neural radiance field: a network mapping 3D position and view direction to color and density, rendered by volume rendering.

Non-maximum suppression (NMS)

Removing overlapping detections by keeping the highest-scoring box and discarding others with IoU above a threshold.

Normalization (images)

Scaling pixel values, for example to [0, 1] or by subtracting the dataset mean and dividing by its std. It must match between training and inference.

O

Object detection

Finding objects in an image and predicting a class and bounding box for each.

ONNX

An open format for exporting trained models to run in many runtimes.

Opening / closing

Erosion then dilation (removes small specks), and dilation then erosion (fills small holes).

Optical flow

The apparent motion of each pixel between consecutive frames.

ORB

Oriented FAST and Rotated BRIEF: a fast, free binary feature detector and descriptor.

Otsu's method

Chooses a global threshold that maximizes the between-class variance of the two resulting pixel groups.

P

Panoptic segmentation

Combines semantic segmentation of "stuff" (sky, road) with instance segmentation of "things" (cars, people).

Patch embedding

Splitting an image into fixed-size patches and linearly projecting each into a token for a vision transformer.

Pinhole camera model

An idealized camera that projects 3D points through a single point onto an image plane: \(\mathbf{x} \sim K[R \mid \mathbf{t}]\mathbf{X}\).

PnP (Perspective-n-Point)

Estimating a camera's pose from known 3D points and their 2D projections.

Point cloud

A set of 3D points, often with color or intensity, from LiDAR, depth cameras, or reconstruction.

Pose estimation

Locating keypoints such as body joints. In geometry, it means estimating a camera's or object's position and orientation.

Principal point

Where the optical axis meets the image plane, usually near the image center.

Projection matrix (P)

The 3×4 matrix \(K[R \mid \mathbf{t}]\) mapping homogeneous world points to image points.

Q

Quantization (images)

Mapping continuous intensities to a finite set of levels. In deployment, it means storing weights and activations with fewer bits.

R

RANSAC

Random sample consensus: robustly fits a model by repeatedly fitting minimal random samples and keeping the one with the most inliers.

Receptive field

The region of the input image that influences one unit in a network layer.

Rectification (stereo)

Warping a stereo pair so that corresponding points lie on the same row.

Region proposal network (RPN)

The part of Faster R-CNN that proposes candidate object boxes.

Reprojection error

The pixel distance between an observed point and the projection of its estimated 3D position. It's the standard measure of calibration and reconstruction quality.

RoI Align

Extracts fixed-size features for a region using bilinear sampling, without the quantization of RoI pooling.

S

SAM (Segment Anything Model)

A promptable segmentation foundation model that produces masks from points, boxes, or masks.

Saliency map

An image showing how much each pixel influences a model's output.

Scale space

A family of progressively blurred versions of an image, used to detect features at their natural scale.

Semantic segmentation

Assigning a class label to every pixel, without distinguishing between instances.

SIFT

Scale-Invariant Feature Transform: detects blobs across scale space and describes them with 128-dimensional gradient histograms.

Sobel operator

A 3×3 derivative filter combined with smoothing, used to estimate image gradients.

sRGB

The standard color space for displays and the web, with a defined gamma curve.

Stereo matching

Finding corresponding pixels between two views to compute disparity.

Stride

The step size of a convolution or pooling window. A stride above 1 downsamples.

Structure from motion (SfM)

Recovering camera poses and 3D structure from many unordered images.

Structure tensor

The matrix of summed products of image gradients in a window. Its eigenvalues describe local edge and corner structure.

Structuring element

The small shape (a square, disk, or cross) that defines a morphological operation's neighborhood.

Super-resolution

Reconstructing a higher-resolution image from a lower-resolution one.

T

Thresholding

Turning a grayscale image into a binary one by comparing each pixel with a threshold, which can be global, Otsu, or adaptive.

Transfer learning

Reusing a model trained on one task or dataset as the starting point for another.

Transposed convolution

A learnable upsampling layer, sometimes misleadingly called "deconvolution".

Triangulation

Recovering a 3D point from its projections in two or more calibrated views.

U

U-Net

An encoder-decoder segmentation network with skip connections between matching resolutions.

Undistortion

Removing lens distortion from an image, using calibrated distortion coefficients.

V

Vanishing point

The image point where parallel 3D lines appear to meet.

Viola-Jones

A fast classical face detector that uses Haar-like features, integral images, boosting, and a cascade.

Vision transformer (ViT)

A transformer applied to a sequence of image patches.

Volume rendering

Computing a pixel's color by integrating color and density along a camera ray. It's the core of NeRF.

W

Warping

Moving pixels according to a geometric transformation, usually implemented by inverse mapping plus interpolation.

Watershed

A segmentation algorithm that treats intensity (or a distance transform) as a landscape and floods it from markers, splitting touching objects.