← Back to Machine Learning and Deep Learning

Machine Learning and Deep Learning Glossary

Key terms from the Machine Learning and Deep Learning course, linked to the lesson that introduces each one.

972 terms.

A

ablation study
An ablation study is like reverse engineering a recipe: you start with a complex dish (Rainbow DQN with all improvements) and remove ingredients one at a time to see which ones actually matter.
Lesson 2236 — Ablation Studies: Which Improvements Matter Most
Above the line
Model is underconfident (predicts lower probabilities than observed)
Lesson 489 — Calibration Plots and Reliability DiagramsLesson 530 — Reliability Diagrams
Absolute scoring
Absolute scoring is efficient—one evaluation per output—and provides interpretable numeric feedback.
Lesson 3162 — Pairwise Comparison vs Absolute Scoring
Accuracy becomes misleading
High accuracy doesn't mean your model is actually useful
Lesson 242 — Class Imbalance Introduction
Accuracy Loss
Accuracy Loss is your usual objective (cross-entropy, MSE, etc.)
Lesson 3310 — Fairness Constraints During Training
action
An action is a choice the agent can make.
Lesson 2134 — States, Actions, and State Spaces
action space
An action space is the complete set of operations an agent can perform—its "toolbox." A tool registry is the system that catalogs these tools with structured descriptions so the agent knows what's available and how to use each one.
Lesson 2062 — Action Space and Tool RegistryLesson 2134 — States, Actions, and State Spaces
actions
Your actions are driving to connected cities or refueling.
Lesson 2083 — Planning in AI Agents: Problem Formulation
activation
Each layer's output is called an activation or intermediate activation.
Lesson 609 — Forward Pass Through Multi-Layer Networks
Activation atlases
Imagine exploring a vast museum where every room represents what a neural network layer "sees." Activation atlases are exactly that—comprehensive maps of learned representations created by collecting millions of neuron activations, clustering them by similarity, and visualizing what each cluster represents.
Lesson 3272 — Activation Atlases and Feature Spaces
Actor
In actor-critic methods, the actor is a neural network that directly represents your policy—it's the component that decides what action to take given a state.
Lesson 2275 — From Pure Policy Gradients to Actor-CriticLesson 2277 — The Actor: Parameterized Policy Networks
Acts as regularization
the batch statistics add noise during training (similar to dropout's effect)
Lesson 752 — Batch Normalization: Core Concept
Acyclic
Acyclic means no circular dependencies—you can't have Task A depending on Task B, which depends on Task C, which depends back on Task A
Lesson 2861 — Directed Acyclic Graphs (DAGs)
AdaBound
AdaMax and AdaBound are two clever variants that address specific limitations of standard Adam.
Lesson 709 — AdaMax and AdaBound Variants
additive smoothing
Laplace smoothing (also called additive smoothing) adds a small "pseudocount" to every possible feature-class combination, even those you've never observed.
Lesson 334 — Laplace Smoothing for Zero Probabilities
adjacency matrix
The adjacency matrix is one fundamental representation: a square matrix where rows and columns represent nodes, and cell values indicate whether an edge exists between them.
Lesson 2484 — Graph Representations: Adjacency Matrix
Adjust learning rates
If gradients are consistently large or small, tune accordingly
Lesson 680 — Gradient Norm Monitoring
Advantage
Errors are more balanced—sometimes slightly high, sometimes slightly low—so they tend to cancel out rather than compound. This usually preserves model accuracy better.
Lesson 2627 — Quantization Error and Rounding
advantage function
The advantage function is a clever mathematical trick that tells us how much better (or worse) an action is compared to the average action we'd take in that state.
Lesson 2278 — Advantage Functions in Actor-Critic
adversarial patches
Unlike typical adversarial examples that apply tiny perturbations across an entire image, adversarial patches are localized regions—think stickers or decals—that can be placed anywhere in a scene to cause misclassification.
Lesson 3385 — Adversarial PatchesLesson 3394 — Adversarial Patches
agents
Understanding the distinction between chains, workflows, and agents is crucial for choosing the right architecture.
Lesson 2058 — Agent vs. Chain vs. WorkflowLesson 2876 — Prefect Cloud and Deployment Patterns
AI agent
An AI agent is a system that operates with a degree of autonomy—it observes its environment, makes decisions based on those observations, and takes actions to accomplish specific objectives.
Lesson 2057 — What is an AI Agent?
AI alignment problem
The AI alignment problem is the challenge of ensuring that AI systems pursue the goals and values their designers intend, rather than unintended interpretations or proxy metrics that can lead to harmful outcomes.
Lesson 3425 — What is the AI Alignment Problem?
AlexNet
In 2012, a team led by Alex Krizhevsky submitted a convolutional neural network called AlexNet to the ImageNet Large Scale Visual Recognition Challenge (ILSVRC).
Lesson 890 — AlexNet: The Deep Learning Revolution
ALiBi takes a different approach
instead of adding positional information to token embeddings, it adds a linear penalty directly to the attention scores based on the distance between tokens. The further apart two tokens are, the more negative bias is added to their attention score, making distant tokens slightly less likely to attend strongly to each other.
Lesson 1656 — ALiBi: Attention with Linear Biases
anomaly
An anomaly is a broader concept: any pattern that doesn't conform to expected behavior, even if individual values seem reasonable (like a user profile with age=25 but retirement_status=true).
Lesson 3056 — Outlier and Anomaly Detection in Data
Anomaly Detection
Since autoencoders learn to reconstruct normal training data well, they struggle with unusual inputs. If reconstruction loss is unusually high for a new sample, it's likely an anomaly. This works brilliantly for fraud detection, manufacturing defect identification, or medical imaging anomalies—the autoencoder simply hasn't "seen" these patterns before.
Lesson 1440 — Applications and Limitations of Basic Autoencoders
ANOVA F-statistic
Tests if feature means differ significantly across target classes
Lesson 444 — Feature Selection: Filter Methods
Answer distributions
Most datasets have imbalanced answer frequencies (e.g., "yes" and "no" are very common)
Lesson 1409 — Visual Question Answering Task Definition
Arbitration agent
A higher-level agent (from hierarchical architectures, lesson 2115) makes the final call
Lesson 2116 — Consensus and Voting Mechanisms
Archived
Model is retired but kept for auditing or rollback
Lesson 2832 — Model Staging and Promotion
Arguments
A dictionary or JSON object with parameter names and values (e.g., {"location": "Paris", "unit": "celsius"})
Lesson 1925 — Parsing Function Call Responses
Assistant
Think of it like a play: the system is the director's notes (invisible to the audience but guiding everything), the user is the audience asking questions, and the assistant is the actor performing on stage.
Lesson 1232 — Instruction Format and Template DesignLesson 1752 — Instruction Format and TemplatesLesson 1854 — System vs User vs Assistant Messages
Asymmetric models
Asymmetric models are optimized for query-document pairs with different characteristics.
Lesson 1974 — Asymmetric vs Symmetric Retrieval
Asymmetric retrieval
Asymmetric retrieval is what happens in typical search scenarios: you have a short, incomplete query (like "best pizza recipes") and need to find relevant documents (full recipe articles).
Lesson 1974 — Asymmetric vs Symmetric Retrieval
Asynchronous training
Asynchronous training is more like independent study—workers compute gradients and immediately update a shared parameter server without waiting for others.
Lesson 2708 — Synchronous vs Asynchronous Training
Atrous Spatial Pyramid Pooling
DeepLab often combines multiple atrous rates in parallel (called Atrous Spatial Pyramid Pooling) to capture objects at different scales simultaneously.
Lesson 981 — DeepLab and Atrous Convolutions
Attack Success Rate
Attack Success Rate is your primary metric.
Lesson 3336 — Measuring Privacy Leakage Empirically
Attention rollout
Attention rollout is a technique that combines attention weights across all layers to create a single attention map showing how input tokens influence the final representation.
Lesson 3259 — Attention Rollout and Flow
attention weights
The resulting values α₁, α₂, ..., αₙ are called attention weights.
Lesson 1041 — Softmax Normalization and Attention Weights
AUC
The ROC curve (Receiver Operating Characteristic) and its AUC (Area Under Curve) are popular, but they can be overly optimistic for imbalanced data.
Lesson 379 — Evaluation Metrics for Anomaly Detection
Audio Source Separation
Audio Source Separation is the task of taking a mixed audio signal and separating it back into its constituent sources.
Lesson 2481 — Audio Source Separation
auto wrap policy
An auto wrap policy is a set of rules that tells FSDP: "wrap modules that meet these criteria." PyTorch FSDP provides two main policies:
Lesson 2736 — Auto Wrap Policy in PyTorch FSDP
autoencoder
An autoencoder is an unsupervised neural network that learns to compress data into a compact representation and then reconstruct the original input from that compressed form.
Lesson 378 — Autoencoders for Anomaly DetectionLesson 406 — Autoencoders for Dimensionality ReductionLesson 1429 — What Autoencoders Are and Why They Matter
autoencoder's decoder
The autoencoder's decoder is the bridge that transforms latent representations back into visual reality.
Lesson 1576 — Decoder Consistency and Reconstruction Quality
autoregressive generation
This is called autoregressive generation: each output becomes the next input, creating a chain of predictions that builds the complete sequence.
Lesson 1030 — Inference and Autoregressive Generation
Autoregressive inference
Autoregressive inference means the decoder generates output sequentially: it produces one token, then uses that token as input to generate the next token, then uses both previous tokens to generate the third, and so on.
Lesson 1100 — Autoregressive Inference
autoregressive sampling
This is called autoregressive sampling because each step depends on (regresses on) the model's own previous outputs.
Lesson 1190 — Autoregressive Sampling at Inference
Average Precision
For a single user's recommendation list, Average Precision is the mean of precision values calculated at each position where a relevant item appears.
Lesson 2376 — Mean Average Precision (MAP)

B

Backfilling
Backfilling is computing features for historical data, typically when you:
Lesson 2887 — Feature Materialization and Backfilling
Backtracking
Backtracking means returning to an earlier decision point (a parent node in the tree) to try a different path.
Lesson 1894 — Backtracking and Path Refinement
Backward fill
Backward fill is useful when you know the sequence will continue and want to use future information (e.g., filling hourly data when you have the next hour's reading).
Lesson 433 — Forward Fill and Backward Fill for Time Series
Barriers
Barriers are synchronization points where all processes must "wait" until everyone arrives before continuing.
Lesson 2797 — Synchronization and Barrier Operations
base model
A base model is the raw pretrained transformer that has learned language patterns from massive text corpora through next-token prediction.
Lesson 1227 — Base Models: Pretraining Objective and CapabilitiesLesson 1750 — Base Models vs Instruction-Tuned Models
Base models
Base models are like blank canvases—they predict what comes next based on patterns, excellent for raw completion
Lesson 1233 — When to Use Base vs Instruction-Tuned Models
Base weights
Base weights are stored in low precision (4-bit or 8-bit)
Lesson 1725 — Quantization Basics for Fine-Tuning
baseline
A baseline is any function b(s) that depends only on the state (not the action).
Lesson 2256 — Baselines for Variance Reduction
baseline model
A baseline model is an intentionally simple approach that establishes a minimum performance bar.
Lesson 141 — Baseline Models: Starting Simple
basis
In linear algebra, a basis is a set of linearly independent vectors that span the entire vector space.
Lesson 11 — Basis and Dimension
batch
Instead of computing predictions one-by-one in a loop, we organize multiple inputs as rows in a matrix (called a batch).
Lesson 607 — Batched Forward Propagation
batch learning
Think of it like learning to cook: batch learning is like taking a cooking class once a year where you practice 100 recipes together.
Lesson 132 — Online Learning: Updating Models in Real-Time
Batch Normalization
Contains running statistics and scale/shift parameters that are sensitive to precision loss; often fused with preceding conv layers instead
Lesson 2641 — Quantization of Specific Layer Types
Batch normalization layers
Biases are typically initialized to zero, but the scale parameter may start at one
Lesson 671 — Bias Initialization
Batch size
Larger batches (256+) are common; scale LR proportionally
Lesson 913 — Residual Networks in Practice
Batch-aware caching
Batch-aware caching is the strategy of separating cached from uncached requests, processing only what's necessary, and reassembling the full batch response in the correct order.
Lesson 2923 — Batch-Aware Caching
Batched forward propagation
Batched forward propagation means stacking multiple input samples together and processing them all simultaneously through the same matrix operations.
Lesson 607 — Batched Forward Propagation
Bayesian approach
Instead of one fixed value, you maintain a distribution over possible parameter values. Rather than saying "the weight is 0.73," you say "the weight is probably around 0.73, with this much uncertainty." You get something like "weight ~ Normal(0.73, 0.15)" where the spread represents your confidence.
Lesson 557 — From Frequentist to Bayesian Perspective
Beam A's page table
Beam A's page table is updated to point to the new page; beam B keeps using the shared one
Lesson 2974 — Copy-on-Write for Shared Prefixes
Beam Width Selection
Typical values are 3-10. Larger beams find better captions but cost more computation. Empirically, quality plateaus around width 5-7 for captioning.
Lesson 1407 — Beam Search for Caption Generation
Bellman backup
A Bellman backup is the fundamental operation that updates a value estimate at a state (or state-action pair) by looking one step ahead and combining immediate reward with discounted future values.
Lesson 2156 — Bellman Backup Operations
Bellman Expectation Equation
The Bellman Expectation Equation is a fundamental recursive relationship that breaks down the value function V(s) into two components:
Lesson 2149 — The Bellman Expectation Equation for V
Below the line
Model is overconfident (predicts higher probabilities than observed)
Lesson 489 — Calibration Plots and Reliability DiagramsLesson 530 — Reliability Diagrams
Bernoulli distribution
The Bernoulli distribution describes this random variable with one parameter p (the probability of success).
Lesson 64 — Common Discrete Distributions: Bernoulli and Binomial
Bernoulli Naive Bayes
Calculate the proportion of documents where each feature is present/absent per class
Lesson 335 — Training Naive Bayes: Parameter Estimation
BERTviz
BERTviz is the most popular library for attention visualization.
Lesson 3261 — Attention Visualization Tools and Libraries
Best practices
Run pilot annotations, measure kappa early, refine guidelines when agreement is low, and use gold standard examples to maintain consistency across annotators.
Lesson 3178 — Annotation Quality and Inter-Rater Agreement
Beta-Binomial conjugacy
If you have a Beta prior on probability and observe Binomial data (coin flips), the posterior is also Beta
Lesson 580 — Conjugate Priors and Analytical Posteriors
Better attention
The cross-attention mechanism lets each word directly query any image patch, just like the visual attention mechanisms you learned, but more flexible.
Lesson 1408 — Transformer-Based Image Captioning
Better disambiguation
Words with multiple meanings are easier to understand with full context
Lesson 1186 — Left-to-Right vs Bidirectional Context
Better final convergence
The gentle final approach helps find better local minima
Lesson 717 — Cosine Annealing
Better generalization
The model learns relationships independent of absolute position
Lesson 1087 — Relative Positional Encodings in Transformers
Better learning
The model focuses on high-level structure, not pixel noise
Lesson 1567 — Latent Space Properties and Dimensionality
Better Memory Utilization
Traditional serving pre-allocates contiguous memory for the full KV cache, wasting space when sequences vary in length. vLLM's paged attention uses memory blocks on-demand, like an operating system manages RAM. This efficiency means you can fit more concurrent requests in the same GPU memory budget.
Lesson 2979 — Performance Characteristics of vLLM
Better semantic integrity
Each chunk is more likely to be self-contained and meaningful
Lesson 1987 — Paragraph-Based Chunking
Bi-encoder
A bi-encoder is an embedding model architecture that uses two independent neural networks (or the same network twice) to convert text into dense vector representations.
Lesson 1334 — Late Interaction Models (ColBERT)Lesson 1951 — Embedding Models: Bi-Encoders for Retrieval
BiLSTM handles local context
By processing text bidirectionally, it captures rich features about each token based on surrounding words. For "Apple Inc.", it learns that "Apple" here is business-related, not fruit-related.
Lesson 1291 — BiLSTM-CRF Architecture for NER
Binary Relevance
Binary Relevance is the simplest approach to handle this: you create a separate yes/no classifier for each label.
Lesson 550 — Problem Transformation: Binary Relevance
BIOES scheme
The BIOES scheme is more explicit with five tags:
Lesson 1288 — NER Tag Schemes: IOB and BIOES
Blackboard architecture
A shared workspace where agents post findings that others can read
Lesson 2120 — Shared Context and Memory in Multi-Agent Systems
Block patterns
The model groups related concepts together, showing it understands phrase boundaries or semantic clusters.
Lesson 1059 — Understanding Attention Weight Visualization
Block-Level Wrapping
Wrap logical modules (e.g., entire Transformer blocks containing attention + FFN). Balances memory and communication—parameters for the whole block are gathered once, used for multiple operations, then discarded.
Lesson 2735 — Unit vs Full Shard Wrapping Strategies
Blurriness
The decoder averages out fine details it cannot precisely reconstruct
Lesson 1576 — Decoder Consistency and Reconstruction Quality
BM25
BM25 (Best Match 25) is a probabilistic ranking function used in traditional information retrieval.
Lesson 1998 — Keyword Search Fundamentals: BM25
Bootstrapping
Bootstrapping means the critic doesn't need complete trajectories.
Lesson 2280 — Temporal Difference Learning in the Critic
Both simultaneously
The most challenging scenario requiring retraining and data collection
Lesson 3041 — Concept Drift vs Data Drift
bottleneck
In an autoencoder, the bottleneck is the narrowest layer—typically in the middle—where your data gets squeezed into far fewer dimensions than the original input.
Lesson 677 — Gradient Flow Analysis Through Network DepthLesson 1431 — The Bottleneck and Latent Space
Business utility
A 1-hour-ahead forecast serves different needs than a 1-month-ahead forecast
Lesson 2395 — Forecasting Horizon and Evaluation Windows

C

C hyperparameter
The C hyperparameter is your control knob for this trade-off.
Lesson 273 — The C Hyperparameter
Caching layers
Caching layers are empty (KV cache blocks, result caches)
Lesson 3009 — Model Warmup and Cold Start Optimization
Calibration
Calibration refers to how well the reward model's confidence matches actual quality.
Lesson 1784 — Calibration and Score Distributions
canary deployment
A canary deployment is a progressive rollout strategy where you route a small percentage of production traffic (e.g., 5%) to your new model while the majority continues using the existing model.
Lesson 3084 — Canary Deployment
Captures non-linearity
A linear model can now treat different age ranges differently without polynomial features
Lesson 441 — Binning and Discretization Techniques
Carbon-aware scheduling
Carbon-aware scheduling means timing your model training to run when the grid is cleanest.
Lesson 3472 — Carbon-Aware Training and Scheduling
cardinality
ResNeXt extends ResNet by introducing a new dimension called cardinality — the number of parallel transformation paths within a block.
Lesson 912 — ResNeXt: Aggregated Residual Transformations
Catastrophic forgetting
The model might optimize for reward but lose its grammar, coherence, or factual knowledge
Lesson 1791 — The Trust Region Constraint
CatBoost
CatBoost is often the slowest during training because it handles categorical features natively with more sophisticated preprocessing.
Lesson 320 — Comparing Boosting Libraries: XGBoost vs LightGBM vs CatBoost
Categorical Cross-Entropy
Categorical Cross-Entropy is its natural extension to multiple classes (3 or more).
Lesson 617 — Categorical Cross-Entropy Loss
Causal
The model only looks backward in time (never into the future), essential for real-time generation
Lesson 2468 — Neural Vocoders: WaveNet
Centering
The teacher's outputs are normalized by subtracting a running average of previous batch centers. Think of it like preventing the model from always predicting the same "safe" answer by shifting the origin dynamically.
Lesson 2567 — DINO: Self-Distillation with No Labels
Centralized store
A single vector database or knowledge graph all agents query and update
Lesson 2120 — Shared Context and Memory in Multi-Agent Systems
chain
A chain is a predetermined sequence of steps executed in order, like a recipe.
Lesson 2058 — Agent vs. Chain vs. Workflow
Channel shuffle
Channel shuffle is an elegant operation that mixes information across groups without expensive computation.
Lesson 923 — ShuffleNet: Channel Shuffle Operations
Character-level
Character-level is more human-intuitive but fragile with unexpected inputs.
Lesson 1644 — Byte-Level vs Character-Level Tokenization
characteristic equation
The characteristic equation is our main tool.
Lesson 17 — Computing Eigenvalues and Eigenvectors
Circuits
Circuits are how features connect and compute together.
Lesson 3266 — Circuits vs Features in Neural Networks
Class 0
Often called the "negative class" (e.g., not spam, disease absent, transaction legitimate)
Lesson 236 — Binary Classification Setup
Class 1
Often called the "positive class" (e.g., spam, disease present, transaction fraudulent)
Lesson 236 — Binary Classification Setup
Class imbalance effects
99% accuracy means nothing if your model just predicts "negative" for everything in a 99:1 imbalanced dataset
Lesson 3128 — Why Aggregate Metrics Hide Problems
Classification
Predict a category → "This email is spam" or "This tumor is benign"
Lesson 235 — What is Classification?
Classification and Escalation
Not every anomaly is an incident. Define severity levels (e.g., critical: immediate harm to users; moderate: degraded performance; low: edge case failures) and clear escalation paths to appropriate stakeholders—technical teams, legal, communications, ethics boards.
Lesson 3535 — Incident Response and Management
Clear preference signal
The chosen response should be meaningfully better than the rejected one. Ambiguous pairs where both are equally good or bad provide weak learning signal.
Lesson 1810 — Preference Dataset Requirements for DPO
CLIP as a vision encoder
CLIP as a vision encoder is particularly powerful because it was pretrained to align image and text representations in the same embedding space.
Lesson 1416 — Vision Encoders for Multimodal LLMs
Clipped identity
Gradient is 1 when |w| < 1, else 0
Lesson 2656 — Binarization Training Techniques
Clustering
Clustering is a core unsupervised learning technique that groups similar data points together based on their features alone.
Lesson 337 — What is Clustering?
CNNs
CNNs are great at local feature extraction but struggle with long-range dependencies
Lesson 2457 — Conformer Architecture for ASR
collate function
A collate function is a custom function you pass to DataLoader that controls exactly how individual samples get combined into a batch.
Lesson 818 — Collate Functions: Custom Batch Creation
column
Each column represents one key position (what's being "looked at")
Lesson 1053 — The Attention Score Matrix
column space
Remember that the column space is all possible linear combinations of a matrix's columns.
Lesson 13 — Rank of a Matrix
Completeness
The model might omit expected fields
Lesson 1913 — Native JSON Mode in Modern LLMs
component
In Kubeflow, a component is a self-contained unit of work—like preprocessing data, training a model, or evaluating results.
Lesson 2878 — Building Kubeflow Pipeline Components
Components
Each Gaussian distribution (you learned this in "Gaussian Distribution as Cluster Model") represents one "ingredient"
Lesson 365 — Mixture Model Definition
Compounding errors
As models are trained on tasks we can't fully verify, small misalignments may amplify over time
Lesson 3431 — The Scalable Oversight Problem
Comprehensive evaluation
Comprehensive evaluation means tracking the full constellation of metrics—not just optimizing for one—and ensuring your intervention is a net positive across fairness, accuracy, and other operational constraints.
Lesson 3316 — Evaluating Mitigation Effectiveness
Computation cost
Computing cosine similarity or Euclidean distance across millions of 768-dimensional vectors means billions of floating-point operations
Lesson 1961 — The Curse of Dimensionality in Vector Search
Computational efficiency
Latent representations are typically 4-8× smaller in each dimension. If an image is 512×512 pixels, its latent might be 64×64. That's a 64× reduction in data to process!
Lesson 287 — Gini Impurity as a Splitting CriterionLesson 607 — Batched Forward PropagationLesson 1354 — Swin Transformer: Hierarchical ArchitectureLesson 1565 — From Pixel Space to Latent Space Diffusion
computational graph
A computational graph is PyTorch's behind-the-scenes record of every operation you perform on tensors with requires_grad=True.
Lesson 641 — What is a Computational Graph?Lesson 791 — The Computational Graph
Computational Speed
Mathematical operations are 10-100x faster. When training neural networks with thousands of iterations, this difference means hours versus days.
Lesson 149 — NumPy Arrays vs Python Lists for ML
Compute costs
Computing gradients through backpropagation across all layers is expensive, especially on long sequences.
Lesson 1711 — The Parameter Efficiency Problem in Fine-Tuning
Concept drift
While statistical tests (like those from lesson 3029) can detect data drift, the most direct way to confirm concept drift is watching your model's performance metrics decline over time windows.
Lesson 3039 — Understanding Concept DriftLesson 3044 — Detecting Concept Drift with Model PerformanceLesson 3047 — Root Cause Analysis for Drift
conditioning
This is called conditioning — we're restricting our infinite family of functions to only those that pass through (or near) our observed points.
Lesson 572 — GP Posterior: Conditioning on DataLesson 579 — Exact Inference: Marginalization and Conditioning
Confabulated Reasoning
The model invents plausible-sounding but factually incorrect intermediate steps. Like a student who confidently cites a nonexistent theorem, the model may reference facts, rules, or definitions that don't exist or were misremembered from training data.
Lesson 1874 — Chain-of-Thought Hallucinations and Errors
confidence interval
A confidence interval is a range of values constructed from your sample data that likely contains the true population parameter.
Lesson 87 — Confidence Intervals
Confidence-Based Routing
The model flags low-confidence predictions for human review. If a medical imaging model is 98% confident, it proceeds automatically; at 65% confidence, a radiologist reviews.
Lesson 3491 — Human-in-the-Loop Design Patterns
Confusing correlation with causation
Segment analysis ("model B wins for mobile users!") discovered after testing is exploratory, not confirmatory.
Lesson 3078 — Interpreting A/B Test Results
Conjugacy
Conjugacy means the prior and posterior belong to the same family of distributions.
Lesson 561 — Conjugate Priors and Analytical Posteriors
conjugate priors
But for certain special prior-likelihood pairs, the posterior has the same distributional form as the prior—these are called conjugate priors.
Lesson 580 — Conjugate Priors and Analytical Posteriors
Consistency
The structured thinking process reduces arbitrary judgment variance across similar examples.
Lesson 3166 — Chain-of-Thought Reasoning for Judges
Constraints
Constraints are the rules or limits you must respect while optimizing.
Lesson 93 — What is Mathematical Optimization?
Constraints and restrictions
Constraints and restrictions are explicit rules you embed in your prompt to limit the model's response space and ensure outputs meet your requirements.
Lesson 1849 — Constraints and Restrictions
container
A container is a running instance of that image.
Lesson 2853 — Docker Containers for ML Projects
Containment
Have predefined rollback procedures, model killswitches, or failover to simpler baselines. Sometimes containment means disabling certain features or reverting to human decision-making temporarily.
Lesson 3535 — Incident Response and Management
Context awareness
A recommendation system that assumes high bandwidth and large screens excludes users in low-connectivity regions or those using assistive technologies.
Lesson 3494 — Inclusive Design and Accessibility
Context constraints
Context constraints are your biggest challenge.
Lesson 1902 — Multi-Step Reasoning Trajectories
Context encoding
Context encoding means creating dense vector representations of both the question and potential answer passages.
Lesson 1301 — Context Encoding and Passage Retrieval
Contextual routing
Same query might route to search_vector_db vs. query_sql_database depending on whether the question is about unstructured documents or structured data
Lesson 2074 — Tool Selection Strategy
Continued pretraining
Continued pretraining means taking a pretrained BERT model and running more masked language modeling (MLM) on domain-specific corpora—legal documents, scientific papers, medical records, or financial reports—before your task-specific fine-tuning.
Lesson 1182 — Domain Adaptation with Continued Pretraining
Continuous action spaces
Continuous action spaces are fundamentally different.
Lesson 2315 — Continuous Action Spaces: Fundamentals
Continuous auditing
Continuous auditing means setting up automated systems that regularly recompute the fairness metrics you care about (demographic parity, equalized odds, etc.) and alert you when they degrade beyond acceptable thresholds.
Lesson 3326 — Continuous Auditing and Monitoring
Continuous case
The joint probability density function f(x, y) works similarly, where probabilities come from integrating over regions.
Lesson 69 — Joint Probability Distributions
Continuous quality spectrum
The model learns to denoise across all noise levels—from nearly pure noise to nearly clean images. This comprehensive training coverage ensures it understands the data distribution at multiple scales and levels of detail.
Lesson 1536 — Why Diffusion Models Generate High Quality
Continuous risk monitoring
Continuous risk monitoring means implementing automated systems that constantly evaluate your ML system's health, fairness, security, and alignment with intended use.
Lesson 3537 — Continuous Risk Monitoring
Contrastive loss
Contrastive loss is the key: it's a loss function that explicitly teaches the network to minimize distance between similar pairs (positive pairs) and maximize distance between dissimilar pairs (negative pairs) in the embedding space.
Lesson 2597 — Contrastive Loss for Siamese Networks
ControlNet
ControlNet is an add-on architecture that accepts spatial conditioning signals—images that encode structural information like:
Lesson 1579 — ControlNet and Spatial Conditioning
Convergence behavior changes
The optimization landscape looks "smoother" with less stochastic exploration
Lesson 2709 — Effective Batch Size in Data Parallelism
conversation buffer
A conversation buffer is simply a structured storage of the dialogue history.
Lesson 2098 — Conversation History Management
convex function
A convex function is one where a line segment connecting any two points on its graph lies on or above the curve itself.
Lesson 97 — Convex Functions
convex set
A convex set is a collection of points where, if you pick any two points inside the set, the entire line segment connecting them also lies completely within the set.
Lesson 96 — Convex Sets
coordinate ascent
Mean-field VI optimizes each factor qⱼ(θⱼ) one at a time while holding others fixed—this is called coordinate ascent.
Lesson 587 — Mean-Field Variational Inference
Copy-on-Write
Copy-on-Write is a memory optimization borrowed from operating systems.
Lesson 2974 — Copy-on-Write for Shared Prefixes
Corrigibility
Corrigibility means an AI system remains safely interruptible and modifiable—it cooperates with corrections rather than resisting them.
Lesson 3435 — Power-Seeking Behavior and Corrigibility
Cosine
Cosine is better for high-dimensional vectors where direction matters more than magnitude (like text embeddings).
Lesson 402 — UMAP: Hyperparameters and Their Effects
Cost
Substantial slowdown due to PCIe bandwidth limits. Moving data between CPU and GPU is much slower than GPU-to-GPU communication. Expect 2-5× longer training times.
Lesson 2737 — CPU Offloading in FSDP
Cost and Scale
Hiring qualified annotators is expensive. If you need 100,000 preference pairs to train a robust reward model or DPO policy, you're looking at significant time and budget. Each comparison requires careful reading and judgment.
Lesson 1817 — Limitations of Human Feedback and Motivation for RLAIF
covariance matrix
The covariance matrix is built by applying your kernel function to every pair of points in your dataset.
Lesson 568 — Kernel Functions and the Covariance Matrix
Covariates
Covariates are additional variables that influence your predictions:
Lesson 2421 — Handling Covariates and External Features
Crafting Edge Cases
Red teamers design prompts that sit at the boundary of acceptable behavior—requests that are technically within guidelines but might trigger unsafe outputs. For example, asking for "creative writing about a heist" versus direct instruction to "tell me how to rob a bank."
Lesson 3449 — Manual Red Teaming Techniques
CRF enforces global consistency
The CRF layer looks at the entire sequence of BiLSTM outputs and picks the most coherent label sequence. It prevents impossible transitions like B-PER → I-LOC and ensures entities are properly bounded.
Lesson 1291 — BiLSTM-CRF Architecture for NER
Critic
A value function that estimates how good states or actions are (borrowed from value-based methods like Q-learning)
Lesson 2275 — From Pure Policy Gradients to Actor-Critic
critical points
The critical points are locations where the ground is perfectly flat in all directions—you could place a marble there and it wouldn't roll anywhere.
Lesson 45 — Critical Points and Extrema
Critique prompt design
Critique prompt design is the art of crafting explicit, structured prompts that direct the model's attention toward particular dimensions of quality, making flaws detectable and actionable.
Lesson 1936 — Critique Prompt Design
Cron expressions
Cron expressions are the classic way to define recurring schedules.
Lesson 2874 — Airflow Scheduling and Triggers
Cross-attention
Cross-attention is like looking back at the English text while writing the French translation to find which English words correspond to the French word you're currently generating
Lesson 1078 — Cross-Attention vs. Self-Attention HeadsLesson 1096 — Cross-Attention Mechanism
cross-entropy loss
Log loss, also called cross-entropy loss, measures the quality of probability predictions.
Lesson 466 — Log Loss (Cross-Entropy Loss)
cross-feature patterns
These cross-feature patterns are crucial but hard to hand-engineer.
Lesson 2368 — Deep Learning for Feature Interactions
Cross-modal attention
Cross-modal attention is the bridge that lets one modality query the other.
Lesson 1376 — Cross-Modal Attention Mechanisms
Cross-Modal Attention Layers
Cross-Modal Attention Layers are inserted at regular intervals.
Lesson 1381 — ViLBERT: Dual-Stream Vision-Language Architecture

D

DAG
A DAG is a directed graph with no cycles—you can't follow edges and return to where you started.
Lesson 2488 — Common Graph Types: Trees, DAGs, and Bipartite Graphs
Data characteristics
If input features are predominantly negative, neurons are more vulnerable
Lesson 655 — The Dying ReLU Problem
Data diversity
Data diversity means covering a broad range of tasks, domains, instruction phrasings, and complexity levels.
Lesson 1755 — Data Quality and Diversity
data drift
Input drift (also called data drift or covariate shift) occurs when the statistical distribution of features your model receives in production differs from the distribution it saw during training.
Lesson 3027 — What is Input Drift and Why It Matters
Data freshness
Data freshness refers to how recent your input data is, while latency measures the delay between data generation and availability for inference.
Lesson 3055 — Freshness and Latency Monitoring
Data leakage
Data leakage is the broader phenomenon: any scenario where information from the test set influences training, breaking the fundamental assumption that evaluation data is unseen.
Lesson 3159 — Benchmark Contamination and Data Leakage
data parallelism
Think of it like this: data parallelism is photocopying a book and having multiple readers process different pages simultaneously.
Lesson 2755 — Model Parallelism vs Data Parallelism
Data quality
Data quality refers to how well each instruction-response pair demonstrates the desired behavior.
Lesson 1755 — Data Quality and Diversity
DataFrame
A DataFrame is essentially a collection of Series (one-dimensional labeled arrays) that all share the same index.
Lesson 166 — DataFrames: Two-Dimensional Tabular Data Structures
Datasheets for datasets
Datasheets for datasets are standardized forms that answer critical questions about a dataset's origins, contents, and intended applications—helping practitioners avoid misuse and understand limitations upfront.
Lesson 3516 — Introduction to Datasheets for Datasets
Debug effectively
Narrow down problems while logs and context are fresh
Lesson 3064 — Leading vs Lagging Indicators
Deceptive alignment
Deceptive alignment is the theoretical risk that a model might learn to behave correctly during training not because it genuinely shares our goals, but because it instrumentally recognizes that appearing aligned helps it get deployed—where it can then pursue different objectives.
Lesson 3431 — The Scalable Oversight ProblemLesson 3432 — Deceptive Alignment Risk
decision boundary
In logistic regression, a decision boundary is the invisible line (or surface) that divides your feature space into regions where the model predicts different classes.
Lesson 248 — Decision Boundaries in Logistic Regression
decision stump
A decision stump is a decision tree with only one split—essentially a tree with just a root and two leaves.
Lesson 310 — Implementing AdaBoost with Decision Stumps
decision tree
A decision tree is a flowchart-like model that makes predictions by asking a series of questions about your data.
Lesson 285 — Decision Tree Fundamentals and Intuition
Decoupling
Separate environment interaction from learning. You might collect 4 steps, then perform 1 gradient update—this ratio is a hyperparameter.
Lesson 2245 — Training Loop Structure
deep component
The deep component is a multi-layer neural network (like the NCF architectures you've learned) that learns low-dimensional embeddings and discovers hidden patterns through non-linear transformations.
Lesson 2367 — Wide & Deep Networks for Recommendations
Deeper networks suffer more
The compounding effect across many layers amplifies the problem
Lesson 751 — Why Normalization Matters in Deep Networks
Defense strategies
Some defenses work better against one type than the other, so understanding the threat model is crucial.
Lesson 3379 — Targeted vs Untargeted Attacks
Delimiters
Delimiters are special characters or sequences that act as visual "fences" to separate prompt components.
Lesson 1845 — Delimiters and Formatting Markers
Demographic and protected attribute slices
Demographic and protected attribute slices are subdivisions of your data based on sensitive characteristics like race, gender, age, disability status, or socioeconomic indicators.
Lesson 3130 — Demographic and Protected Attribute Slices
dendrogram
A dendrogram is a tree-like diagram that shows how hierarchical clustering merges data points or clusters at each step.
Lesson 356 — Dendrograms and Tree Representations
dense layer
In a fully connected layer (also called a dense layer), every single neuron connects to every single neuron in the previous layer.
Lesson 597 — Fully Connected Layers: Dense Connections
Dense prediction tasks
Features at multiple resolutions are perfect for segmentation, detection, and other pixel-level tasks
Lesson 1354 — Swin Transformer: Hierarchical Architecture
Deployment Registry
A central system (like MLflow Model Registry or custom database) that records:
Lesson 3093 — Model Version Management
Diagnose weaknesses
Maybe your model is helpful but often inaccurate
Lesson 3167 — Multi-Aspect Evaluation with LLM Judges
diagonal
Think of it like buying furniture: full covariance is custom-made (expensive, perfect fit), diagonal is adjustable (width/height independent), spherical is one-size (cheap, fast), and tied means all rooms get identical furniture (consistent but less flexible).
Lesson 371 — Covariance Structure Constraints
Diagonal Covariance
The simplest approach treats each action dimension independently. Your network outputs one variance per dimension. This means a 3D action space needs 3 variance values. It's computationally cheap but assumes actions don't correlate.
Lesson 2316 — Policy Representation for Continuous Actions
Diagonal patterns
The model focuses on nearby words—common in language where context is local (e.g., adjectives near nouns).
Lesson 1059 — Understanding Attention Weight Visualization
Differencing
Differencing means subtracting the previous observation from the current one:
Lesson 2401 — Differencing and Integration
Diffusion models
Diffusion models are like an artist who starts with a blurry sketch and refines it with hundreds of careful brush strokes—slow, but the final result is often more detailed and realistic
Lesson 1537 — Trade-offs: Sample Quality vs Generation Speed
Direction
a unit vector v (where to point)
Lesson 761 — Weight Normalization
discount factor γ
The discount factor γ (gamma) is a number between 0 and 1 that controls this trade-off.
Lesson 2138 — Discount Factor Gamma
Discrete reconstruction targets
The model reconstructs patch-level representations, not raw pixels (which are noisy and high-dimensional)
Lesson 2573 — Vision Transformer as Reconstruction Target
discretization
Binning (also called discretization) transforms continuous variables into discrete categories by dividing their range into intervals or "bins." This simple technique helps machine learning models in several ways:
Lesson 441 — Binning and Discretization Techniques
discriminative fine-tuning
Layer-wise learning rate decay (also called discriminative fine-tuning) applies progressively smaller learning rates to earlier layers and larger rates to later, task-specific layers.
Lesson 938 — Learning Rate Considerations for Fine-TuningLesson 1177 — Learning Rate and Layer-Wise Decay
Discriminator
Receives both the image and the condition, learning to judge whether the image is real AND matches the given condition
Lesson 1511 — Conditional GANs (cGAN)
disentanglement
This separation of factors is called disentanglement.
Lesson 1452 — β-VAE for Disentanglement
distributional hypothesis
The distributional hypothesis is elegantly simple: you shall know a word by the company it keeps.
Lesson 1118 — The Distributional Hypothesis
distributional shift
This is called distributional shift, and it poses a critical challenge to alignment robustness.
Lesson 3434 — Distributional Shift and Alignment Robustness
Diverse representation
Your training data must reflect the populations who will use your system. This means intentionally including data from people with disabilities, varied ages, ethnicities, geographies, and socioeconomic backgrounds.
Lesson 3494 — Inclusive Design and Accessibility
Domain adaptation
Domain adaptation means reusing knowledge from one domain (source) to perform well in another (target), even with limited or no target labels.
Lesson 1295 — Domain Adaptation and Zero-Shot NERLesson 1979 — Domain Adaptation for Embedding Models
domain expert persona
A domain expert persona is a system prompt that positions the model as a specialist in a particular field—like a cardiologist, tax accountant, or software architect.
Lesson 1857 — Domain Expert Personas
Domain mismatch
A model might excel at code but struggle with truthfulness—the average obscures this.
Lesson 3160 — Leaderboards and Aggregate Scores
Domain-specific perplexity evaluation
Domain-specific perplexity evaluation means computing perplexity separately on curated datasets from your target domain, rather than mixing all test data together.
Lesson 3143 — Domain-Specific Perplexity Evaluation
Domain-specific rerankers
Domain-specific rerankers are fine-tuned for particular verticals—medical literature, legal documents, scientific papers, or customer support tickets.
Lesson 2008 — Reranking Model Selection
DQN loss function
The DQN loss function is designed to minimize the TD error across batches of experiences, effectively teaching the network to satisfy the Bellman optimality equation.
Lesson 2212 — DQN Loss Function Derivation
dropout2D
Spatial dropout (also called dropout2D or channel dropout) takes a different approach: instead of randomly zeroing individual values within a feature map, it drops entire feature maps (channels) at once.
Lesson 746 — Spatial Dropout for Convolutional Layers
Dual use
Dual use refers to the reality that AI and machine learning technologies inherently possess the capacity to serve both beneficial and harmful purposes.
Lesson 3457 — What is Dual Use in AI and Machine Learning?
Durability
Once committed, changes are permanent
Lesson 2845 — Delta Lake and Time Travel
Dynamic quantization
Dynamic quantization is like a chef who preps ingredients (weights) in advance but adjusts seasoning (activation quantization) to taste while cooking each dish.
Lesson 2632 — Dynamic vs Static Quantization
Dynamic replanning
Dynamic replanning means the agent monitors execution in real-time, detects deviations from expected outcomes, and regenerates a new plan on the fly.
Lesson 2090 — Dynamic Replanning and Error Recovery

E

Each edge
Each edge represents a dependency (which values feed into which operations)
Lesson 643 — The Chain Rule in Computational Graphs
Each node
Each node represents a value (variable or operation result)
Lesson 643 — The Chain Rule in Computational Graphs
Early layers
Early layers (close to input) are often more sensitive because they extract low-level features where precision matters
Lesson 2628 — Where to Apply Quantization in a Model
Early stopping
Early stopping is your safety mechanism—it monitors how well your model performs on a validation set during training and stops adding trees when performance stops improving.
Lesson 319 — Early Stopping and Monitoring in Boosting
edge list
An edge list is simply a list of all edges as pairs:
Lesson 2485 — Graph Representations: Adjacency List and Edge List
Edges
Edges are directed arrows showing which task must complete before another can start
Lesson 2861 — Directed Acyclic Graphs (DAGs)
effective batch size
The effective batch size is the total amount of data processed before gradients are averaged and weights are updated — it's the sum of all workers' local batch sizes.
Lesson 2709 — Effective Batch Size in Data ParallelismLesson 2728 — DDP Debugging and Common PitfallsLesson 2783 — Effective Batch Size vs Physical Batch SizeLesson 2785 — Learning Rate Scaling with Gradient Accumulation
effective receptive field
The effective receptive field is the actual region where gradients are significant enough to matter during training.
Lesson 885 — Effective vs Theoretical Receptive Fields
Efficiency
Swin is typically more memory-efficient because attention is strictly local (within windows). PVT computes global attention but reduces the spatial dimension of keys/values, making it moderately more expensive.
Lesson 1359 — Comparing Hierarchical ViT Architectures
Empirical Bayes
Empirical Bayes is the approach where we treat these hyperparameters as tunable parameters rather than choosing them subjectively or using full hierarchical Bayes (which would put priors on the hyperparameters too).
Lesson 564 — Hyperparameters and Evidence Approximation
Encoder
Takes input data and compresses it into a smaller representation (called the "latent code" or "bottleneck")
Lesson 1429 — What Autoencoders Are and Why They MatterLesson 2574 — MAE: Masked Autoencoder Architecture
Encoder uses bidirectional attention
Each token can attend to all other tokens in the input sequence, both before and after its position. This is self-attention without any masking.
Lesson 1104 — Bidirectional vs Causal Attention
ENN
ENN is most conservative, removing only suspicious samples but may not balance classes fully.
Lesson 542 — Resampling: Undersampling the Majority Class
epoch
An epoch is one complete pass through your training dataset.
Lesson 1708 — Training Duration and Convergence
Equal Error Rate
The Equal Error Rate is the point where the false acceptance rate equals the false rejection rate.
Lesson 2482 — Evaluation Metrics for Speaker Tasks
Equalized odds
Equalized odds is more demanding: it requires both equal true positive rates and equal false positive rates across groups simultaneously.
Lesson 3279 — What is Fairness in Machine Learning?Lesson 3284 — Equalized OddsLesson 3297 — Equal Opportunity and Equalized Odds
error
Your error is how far the ball lands from the basket.
Lesson 120 — ML is Optimization, Not Magic
Error analysis by subgroup
Error analysis by subgroup means examining which types of mistakes your model makes for which groups.
Lesson 3322 — Error Analysis by Subgroup
Error attribution
Error attribution is the detective work: identifying which specific decision or component caused the breakdown.
Lesson 2128 — Trajectory Analysis and Error Attribution
Error correction opportunity
If the model makes a small mistake at step 800, it has 200+ more steps to notice and correct it. This self-correcting nature means errors don't compound catastrophically.
Lesson 1536 — Why Diffusion Models Generate High Quality
Error handling
An invalid action (e.g., "open locked door" without a key) requires replanning
Lesson 1905 — ReAct for Interactive Environments
Euclidean distance
Euclidean distance is the default in K-Means — it's the straight-line distance you'd measure with a ruler:
Lesson 344 — Distance Metrics in K-Means
Evaluation mode
Evaluation mode means setting epsilon=0, so your agent always takes the action it believes is best (the greedy action with highest Q-value).
Lesson 2248 — Evaluation and Testing Protocol
event
An event is a set of outcomes—a collection of one or more results we're interested in.
Lesson 53 — Sample Spaces and Events
evidence
Instead of guessing these values, we can learn them from data by maximizing the marginal likelihood (also called evidence).
Lesson 564 — Hyperparameters and Evidence Approximation
Exact inference
Exact inference means computing probabilities of interest without approximation, using two key operations:
Lesson 579 — Exact Inference: Marginalization and Conditioning
Exact match
The predicted entity boundaries and type must match perfectly.
Lesson 1294 — NER Evaluation Metrics
Example thinking
If your SLA is 100ms and inference takes 40ms, your maximum safe timeout is ~50ms (leaving margin for networking and postprocessing).
Lesson 2917 — Batch Size Selection and Timeout Configuration
Executable
The agent can actually perform them
Lesson 2146 — Formulating Real Problems as MDPs
Execution
Execution is the "worker" that actually performs the computational work—running your preprocessing script on a Spark cluster, training your model on GPUs, or evaluating metrics.
Lesson 2860 — Pipeline Orchestration vs Execution
Execution logic
The actual CUDA kernel performing your operation
Lesson 2967 — Custom Plugins and Operators
Expectation violations
The observation doesn't match what the plan predicted (e.g., expected a customer ID, got "not found")
Lesson 2090 — Dynamic Replanning and Error Recovery
Expected Accuracy
The accuracy a random classifier would achieve given the class distributions
Lesson 464 — Cohen's Kappa: Agreement Beyond Chance
Experiment tracking
Experiment tracking means recording everything needed to reproduce and compare your ML experiments:
Lesson 148 — Model Versioning and Experiment Tracking Basics
Expert capacity
Expert capacity is a hard limit on how many tokens a single expert can process in one forward pass.
Lesson 1694 — Expert Capacity and Token Dropping
exploding gradient problem
While the vanishing gradient problem causes gradients to shrink toward zero, the exploding gradient problem is the opposite nightmare: gradients grow exponentially larger as they backpropagate through layers.
Lesson 676 — The Exploding Gradient Problem
Exploitation
Exploitation means choosing actions that maximize immediate reward based on current knowledge—sticking with the best-known Q-values in Q-Learning or SARSA.
Lesson 510 — Bayesian Optimization FundamentalsLesson 2185 — The Exploration-Exploitation Dilemma
Exploration
Exploration means trying actions you've taken less often, or never, to potentially discover superior strategies.
Lesson 2185 — The Exploration-Exploitation Dilemma
exposure bias
This mismatch between training and testing is called exposure bias.
Lesson 1029 — Teacher Forcing in Training
External validators
External validators are independent mechanisms—like code validators, rule engines, databases, or even other AI models—that check whether an LLM's output meets specific quality criteria before accepting it or triggering another refinement round.
Lesson 1943 — External Validators in Refinement Loops

F

F-Beta score
The F-Beta score is a generalization of the F1 score that lets you control this trade-off using a parameter called beta (β).
Lesson 457 — F-Beta Score: Weighted Precision-Recall Trade-off
F1 Score
The harmonic mean balancing precision and recall.
Lesson 1294 — NER Evaluation Metrics
Failure isolation
Failure isolation is valuable (one agent failing doesn't crash the system)
Lesson 2111 — Multi-Agent Systems: Motivation and Use Cases
Failure to progress
The diagonal pattern breaks down, causing garbled speech
Lesson 2467 — Attention Mechanisms in TTS
Fairness
Systems should treat all individuals and groups equitably, avoiding discrimination and bias. This means testing across demographics, monitoring disparate impact, and actively mitigating discovered biases.
Lesson 3487 — Principles of Responsible AI Development
fairness audit
A fairness audit is a structured, methodical examination of a machine learning system to identify, measure, and document potential biases across protected groups and different use cases.
Lesson 3317 — What is a Fairness Audit?
Faithful Chain-of-Thought
Faithful Chain-of-Thought means the reasoning trace is not just plausible—it's actually correct at each step.
Lesson 1872 — Faithful Chain-of-Thought
fake quantization nodes
In QAT, your training loop looks similar to standard training but with a crucial addition: fake quantization nodes are actively participating in both forward and backward passes.
Lesson 2646 — QAT Training Loop Mechanics
False alarm speech
detecting speech where there is none
Lesson 2482 — Evaluation Metrics for Speaker Tasks
Fast comparison
Comparing two dataset versions is just comparing hashes (milliseconds vs. hours)
Lesson 2839 — Content-Addressable Storage for Data
FastAPI
FastAPI is particularly popular for ML serving because it includes automatic API documentation, type validation with Pydantic, and async support for handling multiple concurrent requests efficiently.
Lesson 2894 — REST APIs for Model Serving
Fault tolerance
Fault tolerance means your system detects and recovers from failures automatically.
Lesson 3011 — Fault Tolerance and Graceful Degradation
Feast
Feast (Feature Store) is the most popular open-source option.
Lesson 2890 — Feature Store Tools: Feast, Tecton, and Alternatives
Feature drift
Feature drift refers to changes in individual feature distributions—for example, your user_age feature's mean shifts from 35 to 42 over six months.
Lesson 3028 — Feature Drift vs Covariate Shift
Feature extraction
Feature extraction means you freeze BERT's weights completely.
Lesson 1173 — Fine-Tuning vs Feature Extraction
feature registry
The feature registry is the metadata brain.
Lesson 2883 — Feature Store Architecture Overview
Feature selection
The network automatically identifies which connections matter
Lesson 736 — L1 Regularization for Sparsity
feature store
A feature store is a centralized repository that manages, stores, and serves ML features.
Lesson 2881 — What is a Feature Store and Why It Matters
features
Your job as an ML practitioner is to translate real-world data into meaningful numerical representations called features.
Lesson 117 — The Role of Features and RepresentationsLesson 3266 — Circuits vs Features in Neural Networks
Feedback
Feedback is how observations influence the agent's next decision in the ReAct loop.
Lesson 2063 — Observation Parsing and Feedback
Feedback mechanisms and recourse
Feedback mechanisms and recourse are the essential safety valves that let affected individuals interact with AI systems after deployment—reporting problems, appealing unfair outcomes, and requesting explanations.
Lesson 3495 — Feedback Mechanisms and Recourse
Few-shot NER
Few-shot NER means teaching a model to recognize entities with just a handful of labeled examples.
Lesson 1296 — Few-Shot NER and Prompting Strategies
Few-shot QA
Few-shot QA means showing the model 1-3 example question-answer pairs first, then asking your real question.
Lesson 1310 — QA with Large Language Models
Final classification layers
Final classification layers are sensitive because small changes in logits can flip predictions
Lesson 2628 — Where to Apply Quantization in a Model
final representation of the CLS token
After passing through all Transformer layers, the final representation of the CLS token is used as the image representation
Lesson 1341 — Class Token (CLS Token)
Find representation gaps
Discover if certain demographics are underrepresented in your data
Lesson 3130 — Demographic and Protected Attribute Slices
Fine-tuned extraction
Fine-tuned extraction means you continue training CLIP (or just parts of it) on your specific task data.
Lesson 1401 — Using CLIP as a Feature Extractor
Fine-tuning
Fine-tuning means taking the pretrained encoder, attaching a new task-specific head (like a classification layer), and continuing to train the entire model (or parts of it) on your labeled dataset.
Lesson 1173 — Fine-Tuning vs Feature ExtractionLesson 1953 — RAG vs Fine-Tuning: When to Use EachLesson 2581 — Transfer Learning from Masked Models
fire module
The fire module is SqueezeNet's core innovation, consisting of two layers:
Lesson 924 — SqueezeNet: Fire Modules and Compression
First and Last Layers
The input embedding and final classification layers often need higher precision to preserve accuracy
Lesson 2641 — Quantization of Specific Layer Types
Fixed maximum sequence length
This is the critical constraint. When you train with sequences up to length 512, the model only learns positional embeddings for positions 0-511. At inference time, you cannot process a sequence of length 600—there's no embedding for position 550!
Lesson 1086 — Absolute Positional Embeddings: Advantages and Limitations
Flexibility
PVT's spatial reduction ratio is adjustable per layer, offering fine-grained control. Swin's window size is more rigid but benefits from the elegant shifted-window mechanism.
Lesson 747 — DropConnect and Weight DroppingLesson 1359 — Comparing Hierarchical ViT Architectures
Flows
Flows are the top-level containers—think of them as your entire workflow.
Lesson 2875 — Prefect Architecture and Task API
force plot
A force plot is a visual representation that shows how each feature's SHAP value "pushes" the prediction away from the base value (the average model output across your dataset).
Lesson 3214 — SHAP Force Plots for Individual Predictions
Formal disclosure programs
Formal disclosure programs are structured processes where companies invite security researchers to report vulnerabilities confidentially.
Lesson 3524 — Disclosure Channels and Bug Bounty Programs
Forward fill
Forward fill is more common because it mimics how information flows in time: you only know the past, not the future.
Lesson 433 — Forward Fill and Backward Fill for Time Series
freezing
In neural networks, freezing means preventing certain layers from updating their weights during training.
Lesson 814 — Model Surgery: Freezing Layers and Transfer Learning Setup
Frozen extraction
Frozen extraction means you keep CLIP's weights unchanged and simply pass your data through it to get embeddings.
Lesson 1401 — Using CLIP as a Feature Extractor
Full fine-tuning
Full fine-tuning means updating every single parameter in a pretrained LLM during training on your specific task or dataset.
Lesson 1701 — What Full Fine-Tuning Means for LLMsLesson 1743 — Comparing PEFT Methods: Parameter Count and Performance
function
A function is a rule that takes an input and produces exactly one output.
Lesson 29 — Functions and Continuity
Function calling
Function calling is more like providing a menu of specific actions with predefined parameters.
Lesson 1922 — Function Calling vs JSON Mode
Function name
A string identifying which tool to use (e.g., "get_weather")
Lesson 1923 — Function Schema DefinitionLesson 1925 — Parsing Function Call Responses

G

GANs
GANs are like a skilled artist who creates the entire painting in one confident stroke—fast, but sometimes the details aren't perfect
Lesson 1482 — GANs vs Other Generative ModelsLesson 1537 — Trade-offs: Sample Quality vs Generation Speed
GAT
φ computes attention scores, ⊕ is attention-weighted sum, γ applies final transformation
Lesson 2512 — Message Passing Neural Networks Framework
Gates
Gates are learnable on/off switches that control information flow.
Lesson 1012 — Gates as a Solution to Gradient Flow
gating network
The gating network is a lightweight learned module—typically just a simple linear layer followed by a softmax—that takes a token's representation as input and outputs a probability distribution over all available experts.
Lesson 1690 — Routing Mechanisms in MoE
Gaussian-Gaussian conjugacy
With a Gaussian prior on the mean and Gaussian likelihood, the posterior mean is also Gaussian
Lesson 580 — Conjugate Priors and Analytical Posteriors
GCN
φ is identity with normalization, ⊕ is normalized sum, γ applies weights and activation
Lesson 2512 — Message Passing Neural Networks Framework
Generate heuristics
Output node/edge probabilities indicating which choices are promising
Lesson 2531 — Combinatorial Optimization with GNNs
Generation Quality
The LLM receives only the top-K retrieved chunks as context. If those chunks are poorly sized, the generator either gets irrelevant noise (large chunks) or fragmented information (tiny chunks). Your LLM might hallucinate to fill gaps or produce vague answers because the context window contains incomplete information.
Lesson 1983 — Why Chunking Matters in RAG
generator
In the GAN minimax game you just learned, the generator is the creative player.
Lesson 1471 — Generator Architecture and Role
Generator loss increasing monotonically
The discriminator is winning too easily
Lesson 1502 — Measuring Training Stability
Global behavior
Global behavior is extremely non-linear and high-dimensional
Lesson 3220 — The Local Fidelity Principle
Global Maximum
The absolute highest point everywhere.
Lesson 95 — Local vs Global Optima
Global Minimum
The absolute lowest point across the entire function—the deepest valley in the entire landscape.
Lesson 95 — Local vs Global Optima
global norm clipping
Gradient clipping by norm (also called global norm clipping) works differently than clipping by value.
Lesson 728 — Gradient Clipping by Norm
goal
Your goal is reaching your destination with enough fuel.
Lesson 2083 — Planning in AI Agents: Problem Formulation
golden dataset
A golden dataset is your evaluation bedrock—the unchanging yardstick against which you measure model performance across versions, months, or even years.
Lesson 3117 — What Makes a Dataset Golden
Google Cloud Vertex AI Model Registry
AWS SageMaker Model Registry and Google Cloud Vertex AI Model Registry are fully managed services that integrate seamlessly with their respective cloud ecosystems.
Lesson 2836 — Alternative Model Registry Solutions
Graceful degradation
Graceful degradation means you serve requests at reduced capacity rather than failing completely.
Lesson 3011 — Fault Tolerance and Graceful Degradation
gradient
The gradient is a vector of partial derivatives that tells us how our cost function changes in every direction at once.
Lesson 211 — The Gradient: Direction of Steepest Ascent
Gradient alone
Shows where the stage is sensitive to light changes
Lesson 3236 — Gradient × Input Method
gradient checkpointing
Activation checkpointing (also called gradient checkpointing) solves this by making a deliberate trade-off: store less, compute more.
Lesson 2786 — Activation Checkpointing Fundamentals
Gradient flow
The skip connection provides a direct path for gradients during backpropagation, alleviating the vanishing gradient problem
Lesson 903 — Residual Learning Formulation
gradient masking
This approach is called gradient masking or gradient obfuscation.
Lesson 3411 — Gradient Masking and Obfuscation
gradient norm
The gradient norm is a single number that captures the overall magnitude of all gradients in your network.
Lesson 726 — Gradient Norm and When to Clip
Gradients
Gradients are computed on GPU, then optionally offloaded to CPU
Lesson 2737 — CPU Offloading in FSDP
graph
A graph is a mathematical structure that represents relationships between objects.
Lesson 2483 — What Is a Graph? Nodes, Edges, and Basic Terminology
graph Laplacian
The graph Laplacian is the key matrix here, and its spectral properties (its eigendecomposition) reveal fundamental structural information about the graph.
Lesson 2493 — Graph Signal Processing and LaplaciansLesson 2498 — Spectral Graph Theory Basics
GraphSAGE
φ is identity, ⊕ can be mean/max/LSTM, γ concatenates and transforms
Lesson 2512 — Message Passing Neural Networks Framework
greedy action selection
In reinforcement learning, greedy action selection means always picking the action with the highest estimated value according to your current Q-function or value estimates.
Lesson 2186 — Greedy Action Selection and Its Limitations
Grid-based representation
Every spatial location is represented, not just detected objects
Lesson 1386 — Vision Transformers in Vision-Language Models
Ground Truth Labels
The actual correct outcomes. For loan approval audits, this means knowing who would have repaid; for hiring, who would have succeeded in the role. Ground truth can be tricky when historical data itself reflects bias.
Lesson 3319 — Data Collection for Audits
Grounding
Grounding means constraining the planning process to only reference real, available tools from your tool registry.
Lesson 2094 — Grounding Plans in Available Tools
Group fairness
Group fairness is easier to measure and enforce—you can check approval rates, error rates, or prediction distributions across groups.
Lesson 3281 — Group Fairness vs Individual Fairness
Grouped-Query Attention
Grouped-Query Attention is the middle ground: divide query heads into groups, where each group shares one K/V head.
Lesson 1610 — Multi-Query and Grouped-Query Attention
groups
Think of W&B's organizational system like a filing cabinet: projects are drawers, groups are folders within drawers, tags are colored sticky notes, and notes are detailed labels.
Lesson 2816 — W&B Run Management and Organization
Guardrail metrics
Guardrail metrics are protective measurements that ensure your deployment doesn't cause collateral damage, even if your target metrics improve.
Lesson 3063 — Guardrail Metrics in Production

H

hallucination
Even within their training data, LLMs sometimes generate plausible-sounding but completely fabricated information—a phenomenon called hallucination.
Lesson 1945 — What RAG Solves: Knowledge Cutoff and Hallucination
Hamming Loss
The fraction of labels incorrectly predicted (false positives + false negatives divided by total labels). Lower is better—it captures partial mistakes.
Lesson 554 — Multi-Label Evaluation Metrics
Hard negative mining
Hard negative mining is the practice of deliberately selecting negative samples that are similar to your positive examples, rather than picking them randomly.
Lesson 2545 — Hard Negative Mining
Hard negatives
Hard negatives are examples that are close to your anchor in embedding space but should be pushed away.
Lesson 1976 — Hard Negatives in Retrieval TrainingLesson 2599 — Hard Negative Mining
Hardware
Multi-GPU setups are often essential for models beyond a few billion parameters
Lesson 1701 — What Full Fine-Tuning Means for LLMs
Healthcare
Patient trials are limited and high-stakes
Lesson 2336 — When to Use Model-Based RL: Sample Efficiency Trade-offs
Hermes
Models like Functionary and Hermes are specifically fine-tuned for function calling and work well locally.
Lesson 1929 — Function Calling with Local Models
Hidden biases
The model might reach correct answers through problematic shortcuts
Lesson 1872 — Faithful Chain-of-Thought
Hidden layer with linear activations
Still just straight lines, no matter how many layers you stack (linear combinations of linear functions are still linear)
Lesson 595 — Why Hidden Layers Matter: Universal Approximation
High bias
the model makes strong assumptions by averaging over many points
Lesson 324 — Choosing K: The Bias-Variance Tradeoff
High dimensions
Sometimes optimizing one coordinate at a time is simpler than computing the full gradient
Lesson 109 — Coordinate Descent
High-cardinality
High-cardinality means a categorical variable has many unique values, making standard one-hot encoding impractical.
Lesson 421 — Handling High-Cardinality Categories
High-dimensional actions
Computing max over millions of Q-values is expensive
Lesson 2249 — From Value Functions to Policies
High-stakes decisions
High-stakes decisions are involved (hiring, lending, criminal justice)
Lesson 3325 — External and Third-Party Audits
Higher learning rates become safe
the normalized distribution is more stable
Lesson 752 — Batch Normalization: Core Concept
higher-order derivative
A higher-order derivative is what you get when you differentiate again.
Lesson 39 — Higher-Order Derivatives
Historical bias exists in labels
The ground truth itself may be discriminatory (e.g., past hiring decisions were biased)
Lesson 3282 — Demographic Parity (Statistical Parity)
Holm's Method
A less conservative step-down procedure that adjusts thresholds sequentially based on ranked p-values.
Lesson 92 — Multiple Testing Correction
Homogeneous
Citation network (all nodes are papers, all edges are citations)
Lesson 2489 — Homogeneous vs Heterogeneous Graphs
Horizontal patterns
Consistent direction means monotonic relationship
Lesson 3213 — SHAP Summary Plots and Feature Importance
Huber loss
Huber loss is a hybrid that acts like MSE for small errors (smooth gradients) and like MAE for large errors (outlier protection):
Lesson 474 — Huber Loss and Robust MetricsLesson 615 — Mean Absolute Error and Huber Loss
Human Override Mechanisms
Automated decisions are made but can be contested or overridden by users or operators who see context the model missed.
Lesson 3491 — Human-in-the-Loop Design Patterns
Hyperparameters
Hyperparameters are like the oven temperature and baking time you set beforehand—you control these directly
Lesson 189 — Parameters vs Hyperparameters
hyperplane
In higher dimensions (which we can't visualize), it's called a hyperplane—but the idea stays the same: a flat boundary that cleanly divides your classes.
Lesson 267 — Linear Separability and Geometric Intuition

I

Idempotency
Idempotency means running a task multiple times produces the same result.
Lesson 2880 — Orchestration Best Practices
Ignoring directionality
A significant result in the wrong direction is still a failed experiment.
Lesson 3078 — Interpreting A/B Test Results
image
An image is the blueprint (read-only template).
Lesson 2853 — Docker Containers for ML Projects
ImageNet
ImageNet is a massive image dataset containing over 14 million labeled images spanning 20,000+ categories.
Lesson 932 — ImageNet and the Data Revolution
Images
Images are continuous, high-dimensional arrays of pixels with spatial structure
Lesson 1374 — Vision-Language Alignment Problem
Imbalanced data
Imbalanced data means some classes have many more examples than others.
Lesson 826 — Handling Imbalanced Data in DataLoaders
Immediate backfilling
A new waiting request instantly fills the freed slot in the very next iteration
Lesson 2983 — Continuous Batching Core Concept
Immutability
Immutability is crucial—never modify a published version in place.
Lesson 3122 — Versioning and Dataset Maintenance
Imperceptibility
Changes are typically bounded by a small ε (epsilon) value, making them undetectable to humans
Lesson 3375 — What Are Adversarial Examples?
Improved quality
The discriminator learns richer, class-specific features
Lesson 1495 — Auxiliary Classifier GAN (AC-GAN)
In-context learning
The ability to learn new tasks from examples in the prompt itself
Lesson 1628 — Emergent Abilities and Phase Transitions
Inactive states
Inactive states are temporarily moved to slower CPU memory
Lesson 1730 — Paged Optimizers for Memory Management
Inductive bias
Inductive bias refers to the assumptions a model architecture makes about the data before seeing it.
Lesson 1345 — Inductive Bias Differences
inertia
The metric is called Within-Cluster Sum of Squares (WCSS), also known as inertia.
Lesson 339 — K-Means Objective Function
Inference
Inference is like running that factory every day: each individual prediction uses far less energy than training, but when you serve billions of predictions to millions of users continuously, the cumulative impact can dwarf the training cost.
Lesson 3471 — Training vs Inference Environmental Costs
Inference Speedup
Combining reduced computation with smaller memory footprints means faster predictions. This matters for real-time applications like autonomous driving or video processing.
Lesson 2666 — Why Prune: Benefits and Trade-offs
infinity norm
The infinity norm is simply the largest absolute value among all components.
Lesson 4 — Vector Norms and Distance Metrics
InfoNCE loss
The InfoNCE loss is the mathematical heart of this framework:
Lesson 2547 — Contrastive Learning Framework and InfoNCE Loss
Informed consent
Informed consent means users understand what data you're collecting, why, how it will be used, and what risks exist.
Lesson 3492 — Consent and Data Practices
Infrastructure Blocks
Infrastructure Blocks are reusable configuration templates stored in Prefect Cloud.
Lesson 2876 — Prefect Cloud and Deployment Patterns
initialization
This choice—called initialization—matters far more than you might expect.
Lesson 340 — Initialization Methods
injectivity
This property is called injectivity—different inputs always produce different outputs.
Lesson 2513 — GIN: Graph Isomorphism Networks
inner loop
The inner loop is where your model learns a specific task.
Lesson 2609 — MAML's Inner and Outer Loop
Input
A natural language instruction ("Summarize this article," "Explain quantum physics simply," "Write a poem about spring")
Lesson 1316 — Fine-Tuning for SummarizationLesson 1749 — What Is Instruction Tuning?
Input alone
Shows where light is currently shining
Lesson 3236 — Gradient × Input Method
Input encoding
Historical values are tokenized with positional encodings that preserve temporal ordering
Lesson 2424 — TimeGPT Architecture and Pretraining Strategy
instruction dataset
An instruction dataset consists of (instruction, response) pairs that teach the model this behavior.
Lesson 1230 — Instruction Dataset Construction
Instruction tuning
Instruction tuning is a fine-tuning phase where you train the pre-trained model on datasets of (instruction, desired response) pairs.
Lesson 1749 — What Is Instruction Tuning?
Instruction-tuned model
Starting from a guided tour route—easier to follow and modify, but some routes are already preferred
Lesson 1236 — Further Fine-Tuning: Starting from Base or Instruction
Instruction-tuned models
Instruction-tuned models (like ChatGPT) are fine-tuned specifically to interpret commands as tasks to execute, not patterns to complete.
Lesson 1228 — Base Model Behavior: Completion vs Following InstructionsLesson 1233 — When to Use Base vs Instruction-Tuned Models
INT4 quantization
INT4 quantization represents each weight using only 4 bits (16 possible values), achieving an 8× compression ratio.
Lesson 2662 — INT4 and Sub-Byte Quantization
Integrity verification
The hash serves as a tamper-proof checksum
Lesson 2839 — Content-Addressable Storage for Data
Intent ambiguity
The same model can classify medical images or power surveillance
Lesson 3458 — Historical Examples of Dual Use Technology
Intentionality
Unlike random noise, adversarial perturbations are specifically optimized to cause misclassification
Lesson 3375 — What Are Adversarial Examples?
Interleaved image-text training
Interleaved image-text training means feeding your model sequences where images and text tokens appear in their natural order, mixed together.
Lesson 1418 — Interleaved Image-Text Training
internal covariate shift
This phenomenon, called internal covariate shift, forces you to use small learning rates and careful weight initialization.
Lesson 751 — Why Normalization Matters in Deep NetworksLesson 752 — Batch Normalization: Core ConceptLesson 873 — Batch Normalization in CNNs
Interpretability
Interpretability refers to the degree to which a human can understand the internal mechanics of a model—how it makes decisions from input to output.
Lesson 736 — L1 Regularization for SparsityLesson 3183 — What is Model Interpretability?
intersection
The intersection is where both circles overlap.
Lesson 947 — Intersection over Union (IoU)
intrinsic dimensionality
In neural networks, many weight matrices have redundant structure — the actual "information content" or intrinsic dimensionality is much lower than the full matrix size suggests.
Lesson 1712 — Low-Rank Matrix Factorization Intuition
Invalidation
Invalidation is critical—stale predictions hurt accuracy.
Lesson 2919 — Result Caching Strategies
IQR
Best when data has outliers or is skewed
Lesson 77 — Descriptive Statistics: Spread and Variability
isotropic Gaussian
An isotropic Gaussian means the distribution is:
Lesson 1529 — Why the Final Distribution is Gaussian
It's directly tied to prediction quality
Lower perplexity means the model assigns higher probability to the actual next tokens
Lesson 3137 — What Perplexity Measures in Language Models
Item-based
The clerk looks at books you've purchased, finds books that were frequently bought by the same groups of people, then recommends those related books.
Lesson 2350 — User-Based vs Item-Based Approaches
Iterative DPO
Iterative DPO means running multiple rounds where you:
Lesson 1816 — Iterative DPO and Online Alignment

J

Jacobian matrix
The Jacobian matrix is a mathematical structure that organizes all these partial derivatives in one place.
Lesson 635 — Jacobian Matrices in Backpropagation
jailbreak
A jailbreak is a carefully crafted input that tricks a large language model into bypassing its built-in safety guardrails—producing harmful, biased, or policy-violating outputs it was explicitly designed to refuse.
Lesson 3413 — What Are Jailbreaks and Why They Matter
Joblib
Joblib is a library designed specifically for efficiently saving and loading Python objects, particularly large NumPy arrays (which is exactly what ML models contain).
Lesson 186 — Saving and Loading Models with Joblib
JSON mode
JSON mode is like giving someone a blank form and saying "fill this out in valid JSON format." The model generates free-form JSON text that you then parse and validate.
Lesson 1922 — Function Calling vs JSON Mode
Just right
The model converges efficiently—fast enough to be practical, stable enough to reliably find a good minimum.
Lesson 101 — Learning Rate and Step Size

K

K-means clustering
K-means clustering is a popular approach: you cluster your weight values into k groups, then use the cluster centroids as your quantization levels.
Lesson 2624 — Uniform vs Non-Uniform Quantization
K-shot
K-shot refers to the number of labeled examples you have per class
Lesson 2584 — N-Way K-Shot Terminology
Kernel
Kernel is the most mathematically precise term.
Lesson 853 — Kernels and Filters: Terminology
kernel function
A kernel function is a mathematical shortcut.
Lesson 279 — The Kernel Function Definition
Kernel launch reduction
Each kernel launch has overhead (~5-20 microseconds). Fusing 3 layers means 3× fewer launches.
Lesson 2959 — Layer and Tensor Fusion
key
Think of it like searching a library: your query is "books about transformers," each book's key is its title and topic tags, and the value is the book's actual content.
Lesson 1051 — Query, Key, Value: The Three Vectors
Keys and values
Keys and values are downsampled (e.g., by 2× or 4×) using convolution or pooling
Lesson 1358 — Pyramid Vision Transformer (PVT)
Keyword-enriched version
The chunk with extracted key terms highlighted
Lesson 1995 — Multi-Representation Chunking
knowledge graph
A knowledge graph is a structured representation of facts as entities (nodes) and relations (edges).
Lesson 2101 — Entity Memory and Knowledge GraphsLesson 2529 — Knowledge Graph Reasoning
Kubeflow
Kubeflow is purpose-built for ML on Kubernetes.
Lesson 2879 — Comparing Orchestration Tools
Kubeflow Pipelines SDK
The kfp Python package lets you author pipeline components, compile pipelines into YAML specifications, and submit them to the Kubeflow Pipelines backend for execution on your Kubernetes cluster.
Lesson 2877 — Kubeflow Pipelines Overview
KV cache eviction
KV cache eviction is the process of selectively removing cached positions when you hit memory limits, keeping only the most valuable information.
Lesson 1678 — KV Cache Eviction Strategies

L

L1 norm
The L1 norm is the sum of the absolute values of all components in a vector.
Lesson 4 — Vector Norms and Distance Metrics
L2 Cache
A 40-80MB buffer sitting between compute cores and VRAM. When the same data is accessed multiple times (like attention weights in transformers), L2 cache can serve it 10-20x faster than VRAM. Cache locality is why compact data layouts matter.
Lesson 2935 — Understanding GPU Memory Hierarchy for Inference
L2 norm
The L2 norm is the square root of the sum of squared components—the "straight-line" distance.
Lesson 4 — Vector Norms and Distance Metrics
Label corrections
A team member fixes 500 mislabeled samples. Your model improves, but last week's experiment is now impossible to recreate.
Lesson 2837 — Why Data Versioning Matters in ML
Lagging indicators
Lagging indicators are the actual business outcomes you care about—revenue, conversion rates, customer retention—but they take days, weeks, or even months to materialize.
Lesson 3064 — Leading vs Lagging Indicators
Language matters
English tolerates lowercasing better than German (where nouns are capitalized)
Lesson 1269 — Tokenizer Normalization and Preprocessing
large margin
A large margin means your boundary has buffer space—it's more likely to generalize well to new, unseen data
Lesson 268 — The Concept of Margin
Larger patches
Larger patches are computationally cheaper but may miss fine-grained patterns.
Lesson 1347 — Resolution and Patch Size Trade-offs
Latency SLOs
Often expressed as percentiles (p50, p95, p99). A p99 < 200ms target means 99% of requests must complete under 200ms.
Lesson 2932 — Service Level Objectives (SLOs) and Budget Allocation
Latent imagination
Latent imagination is the process of planning by "imagining" future trajectories in latent space.
Lesson 2337 — World Models and Latent Imagination
Latent interpolation
Latent interpolation means creating a smooth path between two images in latent space.
Lesson 1577 — Latent Space Interpolation and Editing
Layer freezing
Layer freezing means locking certain layers' weights so they don't update during training, while allowing others to learn from your new data.
Lesson 937 — Layer Freezing StrategiesLesson 941 — Domain Adaptation Challenges
Layer-wise attention analysis
Layer-wise attention analysis means systematically examining how attention weights change across layers, revealing a progression from low-level syntactic patterns to high-level semantic relationships.
Lesson 3258 — Layer-Wise Attention Analysis
lazy learning
This is called lazy learning because the algorithm postpones all computational work until prediction time, unlike eager learning methods (like SVMs or Random Forests) that do heavy computation during training.
Lesson 321 — The Instance-Based Learning Paradigm
Leading indicators
Leading indicators are early warning signals you can measure immediately or soon after deployment—things like prediction latency, confidence scores, input distribution shifts, or user engagement patterns.
Lesson 3064 — Leading vs Lagging Indicators
leaf node
Each leaf node represents a word in your vocabulary
Lesson 1122 — Hierarchical Softmax for Word2Vec
Learned representations
The model discovers its own internal "language" for meaning
Lesson 1035 — Applications: Machine Translation
Learning
Learning is the process of adjusting these parameters (through many attempts) to minimize your misses.
Lesson 120 — ML is Optimization, Not Magic
learning curve
A learning curve is a graph that shows how your model's performance (like accuracy or loss) changes as training progresses—typically measured across epochs or training set size.
Lesson 519 — What Learning Curves Reveal
Learning Rate
Your neural network's step size when updating Q-values. Too high and your agent forgets past knowledge catastrophically; too low and learning crawls. In DQN variants with replay buffers, lower learning rates (e.g., 0.00025 vs 0.001) often work better because samples are reused multiple times.
Lesson 2235 — Hyperparameter Sensitivity in DQN VariantsLesson 2272 — REINFORCE Convergence Properties
Least Squares Criterion
The Least Squares Criterion is simply the principle that the best line is the one that minimizes the sum of squared errors.
Lesson 192 — The Least Squares Criterion
length extrapolation
This is called length extrapolation and solves a major limitation of absolute position encodings.
Lesson 1655 — Rotary Position Embeddings (RoPE)
Less expert knowledge
The model learns patterns from data
Lesson 2452 — End-to-End ASR: Motivation
LightGBM
LightGBM is typically the fastest, especially on large datasets with many rows.
Lesson 320 — Comparing Boosting Libraries: XGBoost vs LightGBM vs CatBoost
likelihood
The likelihood is the probability of all observed targets {y₁, y₂, ..., yₙ} given parameters θ (weights, bias) and inputs X:
Lesson 329 — Bayes' Theorem and Posterior ProbabilityLesson 559 — Likelihood Function for Regression
limit
A limit describes what value a function approaches as its input gets arbitrarily close to some point.
Lesson 30 — Limits: The Foundation of Derivatives
Limitation
LLM requests vary wildly in length. A short prompt might take 100ms while a long generation takes 30 seconds. Round-robin can leave servers idle while others are overloaded.
Lesson 3006 — Load Balancing Strategies for LLM Services
Limited data
Your training set is just a sample, never the complete universe of possibilities. As you learned with sampling distributions, samples contain uncertainty.
Lesson 122 — ML Models as Approximations
Limited Expertise
Many alignment tasks require specialized knowledge (medicine, law, coding). Finding expert annotators for every domain is impractical.
Lesson 1817 — Limitations of Human Feedback and Motivation for RLAIF
linear decision boundary
A linear decision boundary is a straight line (or flat plane in higher dimensions).
Lesson 238 — Decision Boundaries and Separability
Linear independence
Linear independence means vectors provide genuinely different directions—none can be created by combining the others using scalar multiplication and addition.
Lesson 10 — Linear Independence and Span
Linear probing
Linear probing is a diagnostic approach: you freeze the pretrained encoder completely and train only a simple linear classifier on top of the extracted features.
Lesson 2581 — Transfer Learning from Masked Models
Linear separability
Linear separability means you can draw a straight line that perfectly separates all red dots on one side from all blue dots on the other, with no mistakes.
Lesson 267 — Linear Separability and Geometric Intuition
Linearly separable data
Linearly separable data means you can draw a straight line that perfectly separates the classes.
Lesson 238 — Decision Boundaries and Separability
linkage
The linkage criterion ('ward', 'complete', 'average', or 'single')
Lesson 362 — Implementing Hierarchical Clustering with Scikit-learn
Local linearity assumption
Gradients assume your model is locally linear around the input. But neural networks are highly nonlinear. A tiny perturbation might hit a decision boundary, causing gradients to point in misleading directions.
Lesson 3234 — Why Raw Gradients Are Noisy
Local Maximum
A point higher than all nearby points. Picture a hilltop.
Lesson 95 — Local vs Global Optima
Local Minimum
A point where the function value is lower than all nearby points, but not necessarily the lowest overall. Think of it as a valley surrounded by hills.
Lesson 95 — Local vs Global Optima
Local Outlier Factor
The Local Outlier Factor is the workhorse algorithm here.
Lesson 375 — Density-Based Anomaly Detection
locking down exact package versions
In ML, where tiny numerical differences can compound through training, locking down exact package versions is critical for reproducibility.
Lesson 2851 — Managing Python Dependencies with requirements.txt
log-odds
The log-odds (or logit) is just the natural logarithm of the odds:
Lesson 253 — Probabilistic Interpretation and Odds
Logical Leaps
Steps don't follow logically from previous ones. The model might jump from "A is true" to "C is true" without establishing B, creating gaps in the reasoning chain.
Lesson 1874 — Chain-of-Thought Hallucinations and Errors
logits
Given K raw scores (called logits) from your model — one per class — the softmax function does two things:
Lesson 261 — The Softmax Function Definition
Long-term alignment
Long-term alignment means honest critique and pushing through discomfort—better outcomes, but potentially negative immediate feedback.
Lesson 3445 — Short-Term vs Long-Term Alignment
Low bias
the model makes few assumptions and can capture complex patterns
Lesson 324 — Choosing K: The Bias-Variance Tradeoff
Low or negative value
vectors are dissimilar → low relevance
Lesson 1052 — Computing Attention Scores with Dot Products
low-fidelity
These cheaper evaluations are called low-fidelity approximations.
Lesson 516 — Multi-Fidelity Optimization
Low-rank approximation
Low-rank approximation means we keep only the top k singular values and their corresponding columns/rows from U and V^T, then reconstruct an approximate version of the original matrix.
Lesson 24 — Matrix Approximation with SVD
Lower temperatures
Lower temperatures are safer but transfer less nuance.
Lesson 2692 — Practical Distillation: Hyperparameters and Pitfalls

M

Magnitude
a scalar g (how far to go)
Lesson 761 — Weight Normalization
Main effects
The standalone contribution of each feature (diagonal elements)
Lesson 3216 — SHAP Interaction Values
Majority voting
The most common approach—each agent casts one vote, and the option with the most votes wins. Simple but assumes all agents are equally reliable.
Lesson 1880 — Majority Voting ImplementationLesson 2116 — Consensus and Voting Mechanisms
Manhattan distance
Sum of absolute differences along each dimension. Useful when features are independent.
Lesson 2343 — Similarity Metrics for Content Matching
Manual review
A human expert makes the final call
Lesson 3314 — Reject Option Classification
Many-shot prompting
Many-shot prompting is like showing several route examples—now the pattern becomes unmistakable.
Lesson 1838 — One-Shot vs Many-Shot Trade-offs
margin
Recall from the previous lesson that the margin is the distance from the decision boundary to the nearest data point.
Lesson 268 — The Concept of MarginLesson 269 — Hard-Margin SVM Objective
Marginalization
Marginalization is like "summing out" or "integrating out" variables you don't care about.
Lesson 579 — Exact Inference: Marginalization and Conditioning
Markov property
The Markov property means that to compute the image at timestep t, you only need the image from timestep t-1 — not the entire history of how we got there.
Lesson 1525 — The Markov Chain of Noise Addition
Masked models like BERT
Masked models like BERT are trained to fill in missing words when they can see context from both directions.
Lesson 1198 — Why Autoregressive for Generation Tasks
Match human hearing
The mel scale aligns with how we perceive pitch and frequency
Lesson 2464 — Mel Spectrograms as Intermediate Representation
Materialization
Materialization is the ongoing process of computing feature values from raw data and writing them to your feature store—both offline (for training) and online (for serving).
Lesson 2887 — Feature Materialization and Backfilling
matrix
A matrix is a rectangular grid of numbers arranged in rows and columns.
Lesson 1 — Scalars, Vectors, and Matrices: DefinitionsLesson 775 — What is a Tensor?
Matrix dimensions
If W is (n_out × n_in), x is (n_in × 1), and dL/dz is (n_out × 1), then dL/dW is correctly (n_out × n_in).
Lesson 633 — Backpropagation for Fully Connected Layers
matrix factorization
Classical collaborative filtering using matrix factorization (like SVD and ALS) represents users and items as low-dimensional vectors whose dot product approximates ratings.
Lesson 2363 — From Matrix Factorization to Neural Networks
Matrix Multiplication
Matrix Multiplication is the heart of ML computations.
Lesson 158 — Linear Algebra Operations
Matthews Correlation Coefficient
You've learned many classification metrics—accuracy, precision, recall, F1 score, Cohen's Kappa—but Matthews Correlation Coefficient is special because it considers all four cells of the confusion matrix equally.
Lesson 465 — Matthews Correlation Coefficient
Mean shift
Your feature that averaged 100 is now averaging 120
Lesson 3053 — Statistical Summary Monitoring
Measuring alignment
Measuring alignment means creating tests and metrics to assess whether a model genuinely pursues intended goals rather than exploiting loopholes or pursuing unintended instrumental goals.
Lesson 3436 — Measuring and Evaluating Alignment
Median imputation
Median imputation is better when your data has outliers or is skewed.
Lesson 432 — Simple Imputation: Mean, Median, and Mode
Medical diagnosis
Missing a disease (false negative) in one demographic group more often than another is ethically problematic
Lesson 3283 — Equal Opportunity
mel scale
The mel scale is a perceptually-motivated transformation that maps physical frequencies to how humans actually experience pitch.
Lesson 2439 — Mel Scale and Mel-Spectrograms
Memory Reduction
Storing fewer weights directly reduces model size. A pruned model occupies less RAM and disk space—critical for mobile devices or edge deployment where memory is constrained.
Lesson 2666 — Why Prune: Benefits and Trade-offs
Memory savings
A 512×512 RGB image contains 786,432 values. The autoencoder might compress this to a 64×64×4 latent representation—just 16,384 values. That's 48× fewer dimensions to process at each diffusion step.
Lesson 1575 — Computational Benefits of Latent Diffusion
Message passing
Message passing is the mechanism by which agents send and receive information, while communication protocols define the rules and formats for these exchanges.
Lesson 2112 — Agent Communication Protocols and Message Passing
message passing framework
The message passing framework is the fundamental operation that powers most Graph Neural Networks.
Lesson 2496 — The Message Passing Framework
Metadata and lineage tracking
Metadata and lineage tracking means recording detailed information about what data was used, how it was transformed, which models were trained, and when each step occurred throughout your ML pipeline.
Lesson 2862 — Metadata and Lineage Tracking
Metadata enrichment
Metadata enrichment is the practice of tagging each chunk with extra information about its origin and context—like keeping a library card with each page you tear out of a book.
Lesson 1993 — Metadata Enrichment
Method of Moments
The Method of Moments is a parameter estimation technique that works by setting sample statistics (like the mean or variance you calculate from your data) equal to their theoretical counterparts, then solving for the unknown parameters.
Lesson 86 — Method of Moments
min-max scaling
Min-Max Normalization (also called min-max scaling) squeezes all your feature values into a specific range by finding the minimum and maximum values, then rescaling everything proportionally between them.
Lesson 408 — Min-Max Normalization
mini-batch
Stochastic Gradient Descent (SGD) takes a smarter approach: instead of computing the exact gradient from all data, it estimates the gradient using a small random subset called a mini-batch (often 32, 64, or 256 examples).
Lesson 105 — Stochastic Gradient Descent Basics
Mini-batch gradient descent
Mini-batch gradient descent is the "just right" middle ground—it computes gradients on small batches of training examples.
Lesson 684 — Mini-Batch Gradient Descent
miniImageNet
miniImageNet is the most popular few-shot learning benchmark.
Lesson 2594 — Few-Shot Learning Benchmarks
Minimal parameters
Only the prefix vectors are trainable
Lesson 1739 — Prefix Tuning: Prepending Learnable Vectors
Mish activation
A smoother alternative to ReLU that helps gradients flow
Lesson 965 — YOLOv4 and YOLOv5: Speed and Accuracy Advances
Mixed precision quantization
Mixed precision quantization means applying different quantization bit-widths to different parts of your model based on how sensitive each layer is to reduced precision.
Lesson 2629 — Mixed Precision Quantization
Mixout
Mixout is a dropout-inspired technique that randomly keeps some weights at their pretrained values during fine-tuning.
Lesson 1183 — Catastrophic Forgetting and Regularization
ML Development Lifecycle
The ML Development Lifecycle describes this repeating journey through several connected stages.
Lesson 135 — The ML Development Lifecycle Overview
ML pipeline
An ML pipeline is an automated workflow that orchestrates the entire machine learning lifecycle—from data ingestion and preprocessing, through model training and evaluation, to deployment and monitoring.
Lesson 2857 — What is an ML Pipeline?
MMR
MMR is a classic technique that balances relevance with diversity.
Lesson 2009 — Diversity in Reranking
MoCo
Memory is largely independent of negative sample count; runs efficiently on 8 GPUs
Lesson 2557 — SimCLR vs MoCo: Comparative Analysis
Mode
the most frequently occurring value
Lesson 432 — Simple Imputation: Mean, Median, and Mode
mode collapse
This phenomenon is called mode collapse—the model's output distribution collapses from diverse, creative responses down to a narrow set of high-reward patterns.
Lesson 3441 — Mode Collapse and Response Diversity
Mode imputation
Mode imputation is ideal for categorical variables (like "color" or "city") or discrete counts.
Lesson 432 — Simple Imputation: Mean, Median, and Mode
Model artifacts
The trained model files themselves
Lesson 148 — Model Versioning and Experiment Tracking Basics
Model awareness
The model learns to treat these differently—padding tokens don't contribute to loss, <eos> triggers stopping conditions.
Lesson 1648 — Handling Special Tokens
model card
A model card is a structured document that accompanies a machine learning model, providing critical information about its development, capabilities, limitations, and appropriate use cases.
Lesson 3511 — Introduction to Model Cards
model cards
Introduced by Mitchell et al., model cards are standardized reports describing a trained model.
Lesson 3490 — Transparency and Documentation Standards
model complexity curve
A model complexity curve is a plot that shows how both training error and validation error change as you increase the complexity of your model.
Lesson 525 — Model Complexity Curves
Model parallelism
Model parallelism is tearing a single book into chapters and having readers process them sequentially, passing results along.
Lesson 2755 — Model Parallelism vs Data Parallelism
Model Protection
The ML model itself can be kept confidential from unauthorized parties
Lesson 3373 — Trusted Execution Environments
model registry
A model registry is a centralized repository that stores trained machine learning models along with their metadata, versions, and deployment information.
Lesson 2828 — Model Registry Fundamentals
Model serving
Model serving is the process of deploying trained machine learning models into production environments where they can receive input data and return predictions in real time or in batches.
Lesson 2891 — What is Model Serving?
Model versioning
TensorFlow Serving automatically discovers and loads new model versions from a specified directory. Each version is numbered (e.g., /models/my_model/1, /models/my_model/2). By default, it serves the latest version but can maintain multiple versions simultaneously. Clients can request specific versions or use policies like "serve the latest stable version."
Lesson 148 — Model Versioning and Experiment Tracking BasicsLesson 2908 — TensorFlow Serving Architecture
Momentum encoders
Momentum encoders are a clever solution to keep these stored embeddings consistent.
Lesson 2541 — Momentum Encoders and Memory Banks
Monitor training
Watch for signs one network is dominating (discriminator loss near 0 or 1, generator loss exploding)
Lesson 1503 — Learning Rate Balance
More good data beats better algorithms
A linear model with 10,000 clean, diverse examples often beats a neural network with 500 noisy ones.
Lesson 121 — The Data-Centric View of ML
Multi-aspect evaluation
Multi-aspect evaluation means judging outputs across separate, well-defined dimensions:
Lesson 3167 — Multi-Aspect Evaluation with LLM Judges
Multi-class
Multi-class is like choosing your meal from a restaurant menu—you pick one entrée from several options.
Lesson 549 — Multi-Label vs Multi-Class: Key Differences
Multi-label
Multi-label is like choosing toppings for a pizza—you can select multiple toppings or none at all, and each choice is independent.
Lesson 549 — Multi-Label vs Multi-Class: Key Differences
Multi-Model Serving
A single TensorFlow Serving instance can host multiple different models concurrently. It monitors a model base path and automatically loads all models it discovers. Each model gets its own endpoint (e.g., /v1/models/model_a:predict, /v1/models/model_b:predict).
Lesson 2908 — TensorFlow Serving Architecture
Multi-scale inference
Multi-scale inference means running your trained model on the same image at different resolutions (scales), then combining the results.
Lesson 985 — Multi-Scale Inference and Test-Time Augmentation
Multi-tenancy
Multi-tenancy means multiple "tenants" (clients, teams, or model instances) share the same physical hardware—but each must feel like they have dedicated resources.
Lesson 3013 — Multi-Tenancy and Isolation in Shared Infrastructure
Multinomial Naive Bayes
While Gaussian Naive Bayes works well for continuous features (like height or temperature), Multinomial Naive Bayes is designed specifically for count data—features that represent how many times something occurs.
Lesson 332 — Multinomial Naive Bayes for Count Data
Multiple output channels
Multiple output channels means using multiple different filters (kernels) on the same input, producing multiple feature maps simultaneously.
Lesson 859 — Multiple Output Channels

N

N-way
N-way refers to the number of classes you need to distinguish between
Lesson 2584 — N-Way K-Shot Terminology
Named entity recognition
Context on both sides helps identify if "Jordan" is a person or country
Lesson 1024 — Bidirectional LSTMs and GRUs
Nash Equilibrium
Nash Equilibrium is the theoretical endpoint where neither player can improve by changing their strategy alone—they've reached a stable standoff.
Lesson 1474 — Nash Equilibrium in GANs
Natural language inference
Determining if one sentence contradicts or supports another.
Lesson 1148 — The [SEP] Token for Segment Separation
Natural text generation
The model learns to generate coherent sequences without requiring a separate encoder
Lesson 1200 — Decoder-Only Design: Why GPT Diverged from BERT
Nearest Neighbor Baseline
The Nearest Neighbor Baseline is the most straightforward few-shot learning method.
Lesson 2590 — Nearest Neighbor Baseline
Negative
An example from a different class
Lesson 2598 — Triplet Networks and Triplet Loss
Negative determinant
The transformation flips orientation (like mirroring)
Lesson 14 — Determinants and Their Properties
Neighborhood aggregation
Neighborhood aggregation is the fundamental mechanism that lets a node learn from its local graph structure by gathering information from the nodes it's connected to.
Lesson 2492 — Neighborhood Aggregation IntuitionLesson 2495 — Graph Structure and Neighborhood Aggregation
Network architecture sensitivity
The gradient signal must backpropagate through many layers. Each layer's weights, biases, and activation choices distort this signal. Deep networks especially suffer—by the time gradients reach the input, they've been transformed through dozens of nonlinear operations, accumulating numerical artifacts.
Lesson 3234 — Why Raw Gradients Are Noisy
No built-in locality bias
Transformers don't assume nearby patches are related; they learn relationships from data
Lesson 1337 — From CNNs to Vision Transformers
No generalization to longer sequences
Unlike relative positional encoding schemes (not yet covered), absolute embeddings don't inherently understand the relationship between positions. Position 10 and position 11 are encoded as completely independent vectors.
Lesson 1086 — Absolute Positional Embeddings: Advantages and Limitations
No preference learning
It doesn't know whether response A is better than response B for the same query
Lesson 1760 — From Instruction Tuning to Alignment
No states
the environment doesn't change
Lesson 2197 — The Multi-Armed Bandit Problem
No text generation
The model never creates new words or paraphrases
Lesson 1298 — Extractive QA Fundamentals
Nodes
Nodes are individual tasks (data loading, preprocessing, training, evaluation, deployment)
Lesson 2861 — Directed Acyclic Graphs (DAGs)
Noise points
Noise points are labeled -1—these are outliers in regions too sparse to form clusters.
Lesson 354 — Implementing and Evaluating Density-Based Clustering
Nominal categories
Nominal categories are just names or labels with no intrinsic order.
Lesson 418 — Ordinal vs Nominal Categories
Non-linearity
If residuals form a curve, your linear model is trying to fit a curved relationship
Lesson 477 — Residual Analysis and Diagnostic Plots
non-linearly separable
This is called a non-linearly separable problem.
Lesson 592 — Perceptron Limitations: The XOR Problem
Non-separable data
Non-separable data means no straight line works — the classes overlap or interweave.
Lesson 238 — Decision Boundaries and Separability
none
Explanatory responses, when external data is unavailable
Lesson 1930 — Tool Choice Parameters
nonlinear decision boundary
A nonlinear decision boundary is curved or wiggly.
Lesson 238 — Decision Boundaries and Separability
norm
A norm is a function that assigns a non-negative length or magnitude to a vector.
Lesson 4 — Vector Norms and Distance Metrics
Normalization
Normalization is the process of converting text into a canonical form so your tokenizer treats semantically identical inputs consistently.
Lesson 1650 — Normalizing Input Text
Not depending on batch size
statistics are computed per sample, per group
Lesson 759 — Group Normalization
notes
Think of W&B's organizational system like a filing cabinet: projects are drawers, groups are folders within drawers, tags are colored sticky notes, and notes are detailed labels.
Lesson 2816 — W&B Run Management and Organization
Numbers and special characters
Numbers and special characters are particularly inefficient — long numbers might tokenize as individual digits, wasting precious context slots.
Lesson 1651 — Tokenization and Context Window
Numerical stability
Direct matrix inversion can be unstable when XᵀX is nearly singular (non-invertible). A more robust alternative uses np.linalg.lstsq(), which employs QR decomposition or SVD under the hood:
Lesson 202 — Computing the Normal Equation in NumPy

O

object queries
In the Detection Transformer (DETR), object queries are learnable embeddings that act as "slots" or "question marks" for potential objects in an image.
Lesson 1366 — Object Queries and Learned Positional Embeddings
Object structure
If you see part of a wheel, the rest is probably circular
Lesson 2571 — Masked Image Modeling: Core Concept
objective function
The objective function is K-Means' scoring system—it measures how tightly grouped points are within their assigned clusters.
Lesson 339 — K-Means Objective Function
Observability
Observability means instrumenting your system so you can understand why it's behaving a certain way, not just that something is wrong.
Lesson 2868 — Pipeline Monitoring and ObservabilityLesson 3014 — Monitoring and Observability at Scale
Observations
Observations are acoustic features (MFCCs or log-mel spectrograms extracted from audio frames)
Lesson 2449 — Hidden Markov Models for ASR
Offline evaluation
Offline evaluation is fast, cheap, and reproducible—perfect for rapid iteration and comparing dozens of model variants
Lesson 2383 — Offline vs Online Evaluation Trade-offs
Offline metrics
Offline metrics are computed on static, historical datasets—typically your validation or test sets.
Lesson 3059 — What Are Online vs Offline Metrics?
offline store
The offline store is where historical features live for training.
Lesson 2883 — Feature Store Architecture Overview
One-shot
One-shot is computationally cheaper (only one retraining cycle) but often leads to larger accuracy drops, especially at high sparsity levels.
Lesson 2669 — One-Shot vs Iterative Pruning
One-shot prompting
One-shot prompting is like showing someone a single map route and hoping they understand navigation principles.
Lesson 1838 — One-Shot vs Many-Shot Trade-offs
One-shot pruning
One-shot pruning means you identify and remove all weights below your threshold in a single pass.
Lesson 2669 — One-Shot vs Iterative Pruning
one-sided label smoothing
This is called one-sided label smoothing because you typically only smooth the "real" labels, not the fake ones.
Lesson 1505 — Label Smoothing for GANs
Online learning
Online learning means your model updates incrementally with each new example (or small batch) as it arrives, adapting in real-time without needing to retrain from scratch on all historical data.
Lesson 132 — Online Learning: Updating Models in Real-Time
online store
The online store is designed for real-time prediction.
Lesson 2883 — Feature Store Architecture Overview
OpenCLIP
OpenCLIP is an open-source reimplementation that reproduces CLIP's results and goes further.
Lesson 1400 — CLIP Variants and Improvements
operators
Airflow operators are the building blocks that define what actually gets executed in each node of your DAG.
Lesson 2872 — Airflow Operators for ML Workflows
Optimal strategies in some games
Sometimes mixing actions probabilistically is actually optimal (like bluffing in poker)
Lesson 2140 — Policies: Deterministic vs Stochastic
optimistic initial values
In reinforcement learning, optimistic initial values is a clever trick where you initialize all action-value estimates (Q-values) to optimistically high numbers instead of zero or realistic values.
Lesson 2201 — Optimistic Initial Values
Optimistic initialization
Optimistic initialization means setting your initial Q-values deliberately higher than any realistic reward you expect to receive.
Lesson 2193 — Optimistic Initialization
Orchestration
Orchestration is the "conductor" of your pipeline—it decides when and in what order pipeline steps should run, manages dependencies between steps, handles retries on failure, and coordinates the overall workflow.
Lesson 2860 — Pipeline Orchestration vs Execution
Ornstein-Uhlenbeck noise
Ornstein-Uhlenbeck noise is popular because it's temporally correlated (smoother exploration than pure Gaussian).
Lesson 2196 — Exploration in Continuous Action Spaces
orthogonal matrix
An orthogonal matrix is a square matrix whose columns (and rows) are orthonormal vectors—meaning each column has length 1 and is perpendicular to all other columns.
Lesson 21 — Orthogonal Matrices and Their Properties
Orthogonal vectors
Orthogonal vectors are vectors that meet at right angles (90 degrees).
Lesson 20 — Orthogonality and Orthonormal Vectors
orthonormal basis
An orthonormal basis is a set of orthonormal vectors that span a space—like a perfectly aligned, unit-length coordinate system.
Lesson 20 — Orthogonality and Orthonormal Vectors
outcome
An outcome is a single element from the sample space—one specific result.
Lesson 53 — Sample Spaces and Events
Outer loop
The real judges (outer test folds) who never saw your practice sessions
Lesson 498 — Nested Cross-Validation for Hyperparameter Tuning
Outer vs inner loops
The outer loop iterates over episodes (full environment runs), while the inner loop handles individual timesteps within each episode.
Lesson 2245 — Training Loop Structure
outlier
An outlier is a statistically extreme value (like a temperature reading of 500°C when typical values are 20-30°C).
Lesson 3056 — Outlier and Anomaly Detection in Data
over-smoothing
This phenomenon is called over-smoothing.
Lesson 2505 — Multiple Message Passing Layers and Depth
overall silhouette score
The overall silhouette score is the average across all points, ranging from -1 to +1.
Lesson 342 — Silhouette Score
Overlapping chunks
Overlapping chunks means each chunk shares some tokens with its neighbors.
Lesson 1985 — Overlapping Chunks
Oversampling
Oversampling means creating more copies of the minority class samples so the training set becomes more balanced.
Lesson 539 — Resampling: Oversampling the Minority Class

P

padding mask
A padding mask is a binary matrix (same shape as your attention score matrix) that marks which positions are real tokens (1 or True) and which are padding (0 or False).
Lesson 1063 — Padding Masks for Variable-Length Sequences
Page table
A mapping that translates logical block IDs to physical memory locations
Lesson 2971 — Virtual Memory Concepts for LLM Serving
Parallel processing
Parallel processing is possible (multiple agents researching different sources)
Lesson 2111 — Multi-Agent Systems: Motivation and Use Cases
Parameterization
Parameterization means externalizing these decisions into configuration files or command-line arguments.
Lesson 2863 — Parameterization and Configuration
parameters
Parameters are like the final texture and taste that emerge from the baking process—they develop automatically based on what happens in the oven
Lesson 120 — ML is Optimization, Not MagicLesson 189 — Parameters vs HyperparametersLesson 505 — What Are Hyperparameters vs Parameters
Pareto frontier
The Pareto frontier represents the best possible combinations—points where you can't improve fairness without losing accuracy, or vice versa.
Lesson 3315 — Trade-offs Between Fairness and Accuracy
Pareto optimization
Pareto optimization is an alternative: instead of forcing a single score, identify solutions where improving one metric requires sacrificing another.
Lesson 3101 — Multi-Task and Multi-Objective Evaluation
Partial match
Some systems give partial credit when boundaries overlap, even if not exact. This is less common but useful for understanding model behavior during development.
Lesson 1294 — NER Evaluation Metrics
Passage retrieval
Passage retrieval is the step before span prediction.
Lesson 1301 — Context Encoding and Passage Retrieval
path
A path is a sequence of edges connecting two nodes.
Lesson 2487 — Graph Properties: Degree, Connectivity, and Paths
Path filtering
Path filtering is the practice of pre-screening your generated reasoning chains before applying majority voting.
Lesson 1885 — Filtering Low-Quality Paths
Path refinement
Path refinement means learning from failed attempts to make smarter choices when exploring alternatives.
Lesson 1894 — Backtracking and Path Refinement
patience
If performance doesn't improve for a specified number of epochs (called patience), you stop training and restore the model to its best state.
Lesson 832 — Early Stopping Implementation
perceptron
A perceptron is the most basic artificial neuron, designed as a linear classifier.
Lesson 590 — The Perceptron: A Single Artificial Neuron
perceptron learning rule
The perceptron learning rule is an iterative algorithm that teaches a single neuron to classify data correctly.
Lesson 591 — Perceptron Learning Rule: Training a Single Neuron
Performance
The Normal Equation has time complexity O(n³) due to matrix inversion, where n is the number of features. For very large feature sets (thousands+), iterative methods like gradient descent become more efficient.
Lesson 202 — Computing the Normal Equation in NumPy
Performance degradation
The final global model may perform poorly on all clients, especially underrepresented ones
Lesson 3356 — Handling Non-IID Data
persona
A persona is a character or role you assign to the model that governs how it responds, not just what it responds with.
Lesson 1855 — Defining Model Personas
PGD
PGD is essentially BIM with random initialization—instead of starting from the clean image, you start from a random point within the perturbation budget, then iterate.
Lesson 3390 — Basic Iterative Method (BIM) and PGD
pgvector
pgvector is the most popular PostgreSQL extension for vector operations.
Lesson 1967 — Embedding Traditional Databases: pgvector and Extensions
phonemes
In spoken language, phonemes are the smallest distinct units of sound that differentiate meaning.
Lesson 2447 — Phonemes and Linguistic Units
physical batch size
This physical batch size is what actually fits in memory during a single forward-backward pass.
Lesson 2783 — Effective Batch Size vs Physical Batch Size
Physical blocks
Actual GPU memory locations where those blocks are stored
Lesson 2973 — Block Management and Page Tables
Physical memory
The actual GPU memory is divided into fixed-size pages (like apartments)
Lesson 2971 — Virtual Memory Concepts for LLM Serving
Physical-world adversarial examples
Physical-world adversarial examples are designed to remain effective after undergoing transformations like printing, photography, lighting changes, viewing angles, and environmental conditions.
Lesson 3398 — Physical-World Adversarial Examples
Pinecone
Pinecone is a fully managed vector database that handles infrastructure, scaling, and optimization for you.
Lesson 1966 — Vector Database Options: Pinecone, Weaviate, Qdrant
Pipeline changes
Preprocessing code updates (new normalization, augmentation). The data content is identical, but the transformed features differ completely.
Lesson 2837 — Why Data Versioning Matters in ML
Pixel features
Pixel features are simpler and end-to-end trainable, allowing the visual encoder to adapt to the task.
Lesson 1385 — Region Features vs Pixel Features in VL Models
policy
A policy is the strategy your agent follows—it tells the agent what action to take in any given state.
Lesson 2140 — Policies: Deterministic vs Stochastic
Policy improvement
Identify which actions are better than what your current policy suggests
Lesson 2143 — Action-Value Functions: Q-Functions
polynomial kernel
The polynomial kernel is another way to create nonlinear decision boundaries in SVMs, but it works differently.
Lesson 283 — Polynomial Kernel and Degree Selection
polysemantic neurons
Remember from lesson 3269 that neurons often represent multiple unrelated concepts simultaneously—a phenomenon called polysemantic neurons or superposition.
Lesson 3276 — Sparse Autoencoders for Disentanglement
Poor generalization
The model effectively becomes smaller than intended
Lesson 1693 — Load Balancing in MoE
population
Key Takeaway: A population is the complete set you want to understand; a sample is the smaller subset you actually observe.
Lesson 75 — Population vs Sample
Population parameters
Population parameters are the true values (mean, variance, etc.) that describe the entire population.
Lesson 75 — Population vs Sample
Position becomes absolute context
The model treats "the 10th word" differently whether it appears in a 15-word sentence or a 500-word document, even though the local context might be identical.
Lesson 1086 — Absolute Positional Embeddings: Advantages and Limitations
position bias
In practice, however, position bias means the judge favors whichever output appears first (or sometimes last), regardless of actual merit.
Lesson 3164 — Position Bias in LLM Judges
Positive
An example from the same class as the anchor
Lesson 2598 — Triplet Networks and Triplet Loss
Post-deployment validation
Post-deployment validation is the critical monitoring period immediately after deployment where you actively watch for unexpected issues that testing missed.
Lesson 3094 — Post-Deployment Validation
posterior
The posterior is the updated probability that you have the disease after seeing your symptoms.
Lesson 329 — Bayes' Theorem and Posterior Probability
Practical significance
The effect matters for your business
Lesson 3078 — Interpreting A/B Test Results
Pre-filtering
Apply metadata conditions first, then search vectors within that subset (more efficient, but may miss edge cases if the filtered set is small)
Lesson 1968 — Metadata Filtering in Vector Search
Predictable spread
The standard deviation of sample means equals the population standard deviation divided by √n
Lesson 81 — Central Limit Theorem
Prediction Loss
Prediction Loss is your usual objective (cross-entropy, MSE, etc.)
Lesson 3311 — Regularization for Fairness
Predictive mean
The most likely output value, computed using the kernel's covariance between x\* and your training data
Lesson 573 — GP Prediction: Mean and Uncertainty
Prefix tuning
Prefix tuning is like stationing a helpful guide on every floor who can directly assist visitors at each level.
Lesson 1740 — Prompt Tuning vs Prefix Tuning
Premature Conclusions
The model reaches an answer before completing necessary reasoning steps, then backfills justification that appears complete but skips critical verification.
Lesson 1874 — Chain-of-Thought Hallucinations and Errors
Preserving meaning
Preserving meaning is critical—models must avoid hallucinations or semantic drift.
Lesson 1319 — Paraphrasing and Text Simplification
Primacy effects
The first experience disproportionately shapes user perception. A frustrating initial interaction might cause users to avoid the feature later, even after you've fixed the issues.
Lesson 3081 — Long-Term Effects and Novelty Bias
primal formulation
The primal formulation is the original way to state the SVM problem before any mathematical transformations.
Lesson 271 — Primal Formulation of Hard-Margin SVM
primary metric
Your primary metric is the single north star you're optimizing for—the metric that will determine whether variant B "wins." This should align tightly with your core business objective.
Lesson 3073 — Choosing Evaluation Metrics for A/B Tests
prior
The prior is how common a disease is in the population.
Lesson 329 — Bayes' Theorem and Posterior Probability
Private Inference
A model runs inside the enclave, processing user queries without the cloud provider (or even the OS) seeing the input data or outputs
Lesson 3373 — Trusted Execution Environments
Probabilistic outputs
Most ML models output probabilities or confidence scores—"I'm 87% confident this is a cat"—not binary certainties.
Lesson 122 — ML Models as Approximations
Probability Flow ODE
The Probability Flow ODE is a remarkable discovery: there exists a deterministic ordinary differential equation that produces exactly the same marginal distributions as the stochastic SDE, but without any randomness.
Lesson 1561 — Probability Flow ODE
process group
The rank is a unique ID for each process (0, 1, 2, ...), the world size is the total number of processes, and a process group is the communication channel connecting them all.
Lesson 2794 — Distributed Process Groups and Ranks
Production
Model is actively serving predictions
Lesson 2832 — Model Staging and Promotion
Profiling
Profiling means examining what autograd is actually tracking—checking if gradients exist where expected and understanding why they might be missing.
Lesson 800 — Autograd Profiling and Common Pitfalls
projection head
Instead, we add a small MLP—called a projection head—that transforms h into z, and compute the InfoNCE loss on z.
Lesson 2551 — Projection Head Design and Representation Quality
projects
Think of W&B's organizational system like a filing cabinet: projects are drawers, groups are folders within drawers, tags are colored sticky notes, and notes are detailed labels.
Lesson 2816 — W&B Run Management and Organization
Prompt engineering as defense
Prompt engineering as defense means architecting your system prompt with structural boundaries that make it harder for user input to masquerade as system instructions.
Lesson 3423 — Defense: Prompt Engineering Against Injection
prompt template
A prompt template is a reusable prompt structure with placeholders—special markers that indicate where dynamic content should be inserted.
Lesson 1847 — Prompt Templates and Placeholders
Prompt tuning
Prompt tuning is like giving special instructions to visitors at the lobby entrance.
Lesson 1740 — Prompt Tuning vs Prefix Tuning
Provenance tracking
Provenance tracking means recording the complete lineage of your data:
Lesson 1642 — Documenting and Reproducing Data Pipelines
Proxy metrics
Proxy metrics are measurable signals available immediately that correlate with model quality.
Lesson 3018 — Proxy Metrics for Real-Time Monitoring
pseudo-residuals
Fit a tree to residuals: Train a new tree to predict these errors (these are called pseudo-residuals)
Lesson 312 — Gradient Boosting for Regression
Public disclosure
Public disclosure means revealing a vulnerability openly—through blog posts, conference talks, social media, or research papers—before it's fully patched or acknowledged.
Lesson 3526 — Public Disclosure Decisions
Public trust
Public trust is critical to your product's success
Lesson 3325 — External and Third-Party Audits

Q

Q-functions
That's where action-value functions, commonly called Q-functions, come in.
Lesson 2143 — Action-Value Functions: Q-FunctionsLesson 2148 — Action-Value Functions (Q-Functions)
Q-learning
Q-learning is like studying the optimal racing line in theory, even while you drive conservatively.
Lesson 2178 — Q-Learning vs SARSA: Key Differences
Qdrant
Qdrant is a lightweight, Rust-based vector database emphasizing performance and simplicity.
Lesson 1966 — Vector Database Options: Pinecone, Weaviate, Qdrant
quadratic form
A quadratic form is an expression that looks like x^T A x, where:
Lesson 26 — Quadratic Forms
Quality preservation
A well-trained encoder (from VAE training) captures the semantically important features while discarding imperceptible details. The diffusion model learns to generate in this meaningful, compressed space where the structure is already more abstract and easier to model.
Lesson 1565 — From Pixel Space to Latent Space Diffusion
Quantiles
Quantiles are the general term for these division points.
Lesson 78 — Percentiles and Quantiles
quantization error
This loss is called quantization error, and it's determined by how you handle numbers that fall between integer steps.
Lesson 2627 — Quantization Error and Rounding
query
Think of it like searching a library: your query is "books about transformers," each book's key is its title and topic tags, and the value is the book's actual content.
Lesson 1051 — Query, Key, Value: The Three Vectors
Query Set
Unlabeled examples from the same classes that the model must classify after "seeing" the support set. This is your test set for evaluating how well the model generalized from those few support examples.
Lesson 2585 — Support Set vs Query Set
Query transformation
Query transformation means converting a user's natural language question into a machine-executable query format.
Lesson 2021 — Query Transformation for Structured Data
Question Answering
The context/passage is encoded separately from generating the answer
Lesson 1102 — Encoder-Decoder vs Decoder-Only Trade-offs
Queue depth limits
Reject new requests when the queue is full (fail fast)
Lesson 3007 — Request Queuing and Priority Management

R

RAG
RAG is like giving someone a research library and teaching them to look things up on demand
Lesson 1953 — RAG vs Fine-Tuning: When to Use Each
Random undersampling
Random undersampling is fastest but risks losing informative samples.
Lesson 542 — Resampling: Undersampling the Majority Class
random variable
A random variable is simply a numerical outcome of a random process.
Lesson 58 — Random Variables: Discrete and Continuous
rank
The rank is a unique ID for each process (0, 1, 2, ...), the world size is the total number of processes, and a process group is the communication channel connecting them all.
Lesson 2717 — Process Groups and InitializationLesson 2794 — Distributed Process Groups and Ranks
receptive field
A receptive field is the region of the input image that influences the activation of a single neuron in a particular CNN layer.
Lesson 879 — What is a Receptive Field?
reconstruction loss
The reconstruction loss is your primary training signal.
Lesson 1435 — Training Dynamics and Convergence
Recurrent connections
Standard dropout can disrupt temporal dependencies in RNNs. Specialized variants like recurrent dropout are needed instead.
Lesson 750 — When Dropout Helps and When It Doesn't
Red team testing
Red team testing is the practice of deliberately trying to break your model's alignment—finding prompts that cause harmful outputs despite your constitutional principles.
Lesson 1826 — Iterative Refinement and Red Team Testing
reduce
Reduce is less common alone because in data parallelism you typically need the combined result everywhere, not just on one worker.
Lesson 2721 — Broadcast and Reduce Operations
Reduce bias
Judges reasoning about one clear criterion are less likely to conflate issues
Lesson 3167 — Multi-Aspect Evaluation with LLM Judges
Reduced confusion
The model knows exactly what information to use and what operation to perform
Lesson 1843 — Context vs. Task Separation
Reduces co-adaptation
The network can't rely on any single layer always being present
Lesson 748 — Stochastic Depth
Reduces noise
Small fluctuations within a bin are ignored
Lesson 441 — Binning and Discretization Techniques
Redundancy reduction term
Pushes off-diagonal elements toward 0 (dimensions are decorrelated)
Lesson 2565 — Barlow Twins: Redundancy Reduction
reference model
In Direct Preference Optimization (DPO), the reference model is a frozen copy of your starting policy (typically the SFT model).
Lesson 1808 — The Reference Model in DPO
Regularization
Regularization is the practice of adding a penalty for model complexity directly into your loss function.
Lesson 223 — Introduction to Regularization
Regularization effect
The noise from batch statistics acts like a mild regularizer
Lesson 873 — Batch Normalization in CNNs
regularization path
A regularization path is a visualization that shows how each feature's coefficient changes as you sweep through different regularization strengths.
Lesson 232 — Regularization Paths
Regularization strength
Generally provides finer-grained regularization since individual weights are masked
Lesson 747 — DropConnect and Weight Dropping
Relative time distances
The gap between observations matters (1 minute vs 1 week)
Lesson 2417 — Transformers for Time Series Forecasting
reliability diagram
A calibration plot (also called a reliability diagram) does exactly this check for your ML model's probability predictions.
Lesson 489 — Calibration Plots and Reliability Diagrams
Replay Buffer Size
Think of this as your agent's memory capacity. A larger buffer (1M transitions) provides more diverse experiences but uses more RAM and may include outdated policies. Smaller buffers (100K) update faster but risk overfitting to recent experiences. Prioritized replay amplifies this sensitivity—if your buffer is too small, you'll replay the same "important" transitions obsessively.
Lesson 2235 — Hyperparameter Sensitivity in DQN Variants
Reproducibility
The hash uniquely identifies the exact dataset version used
Lesson 2839 — Content-Addressable Storage for Data
Resampling
Resampling is the process of converting data from one temporal resolution to another—like converting hourly temperature readings into daily averages, or filling in monthly sales data to get weekly estimates.
Lesson 2394 — Resampling and Frequency Conversion
Reservation
These tokens are added to the vocabulary explicitly and assigned fixed IDs, often at the beginning or end of the vocabulary range.
Lesson 1648 — Handling Special Tokens
residual
A residual is simply the error your model makes on each data point: the actual value minus the predicted value.
Lesson 190 — Residuals and Prediction ErrorsLesson 527 — Residual Analysis for Regression
residual block
A residual block is the concrete implementation of skip connections in ResNet.
Lesson 904 — The Residual Block Architecture
ResNet-50
A residual block might have [64, 64, 256] filters across its convolutions
Lesson 910 — ResNet Family: 18, 34, 50, 101, 152Lesson 911 — Wide Residual Networks (WRN)
Resource-constrained planning
Resource-constrained planning means designing agent behavior that achieves goals while staying within hard limits on:
Lesson 2093 — Resource-Constrained Planning
Result Storage and Display
Computed metrics are stored with metadata (timestamp, model description, hyperparameters) and displayed on a public leaderboard, often with filtering, sorting, and historical tracking capabilities.
Lesson 3125 — Leaderboards and Evaluation Infrastructure
Retrieval decision making
Retrieval decision making means using the LLM itself to classify whether a query requires external context or can be answered directly from its parametric knowledge.
Lesson 2046 — Retrieval Decision Making
Reveal patterns
Systematic residuals indicate your model is missing something important
Lesson 190 — Residuals and Prediction Errors
Reversibility
Reversibility means your tokenization process preserves enough information to convert tokens back to text exactly as it was.
Lesson 1247 — Reversibility and Detokenization
Reward hacking
Specification gaming (also called reward hacking) occurs when a model discovers and exploits these loopholes, achieving high measured performance while failing at the true underlying goal.
Lesson 1791 — The Trust Region ConstraintLesson 3426 — Specification Gaming and Reward HackingLesson 3431 — The Scalable Oversight Problem
Richer generation context
The LLM sees the full picture, not isolated fragments
Lesson 1994 — Parent-Child Chunking
Right to know
Individuals must be informed when significant decisions are automated
Lesson 3505 — Algorithmic Transparency and Explainability Requirements
risk taxonomy
A risk taxonomy is a structured classification system—think of it as organizing a messy toolbox into labeled drawers.
Lesson 3531 — Risk Identification and Taxonomy
Robustness
The model doesn't overfit to one specific tokenization pattern
Lesson 1263 — Subword Regularization
ROC Curve
The ROC Curve is a graph that shows how well your classifier performs at every possible threshold setting.
Lesson 480 — Receiver Operating Characteristic (ROC) Curve
Role and persona assignment
Role and persona assignment means telling the model who it should act as when generating a response.
Lesson 1848 — Role and Persona Assignment
Role-based agent specialization
Role-based agent specialization means deliberately designing agents with focused capabilities, knowledge, and responsibilities.
Lesson 2114 — Role-Based Agent Specialization
Roles
A "researcher" agent retrieves information while a "writer" agent drafts responses
Lesson 2111 — Multi-Agent Systems: Motivation and Use Cases
rollback mechanism
A rollback mechanism is your emergency exit: a predefined procedure to instantly restore the last known good model version before issues impact more users.
Lesson 3090 — Rollback Mechanisms
Rollout generation
Rollout generation means sampling complete response sequences from your current language model (the policy) given various prompts, then collecting the rewards for each of those generations.
Lesson 1796 — Rollout Generation and Experience Collection
Routing
Routing means using the question itself to decide which source(s) to query.
Lesson 2051 — Routing to Multiple Knowledge Sources
row
Each row represents one query position (what's currently "asking")
Lesson 1053 — The Attention Score Matrix
Rows
Query positions (what the model is currently processing)
Lesson 1059 — Understanding Attention Weight Visualization

S

SAC
SAC is also stable but requires careful tuning of the temperature parameter (α), which controls the exploration-exploitation trade-off.
Lesson 2324 — SAC vs TD3: When to Use Which
Saddle Point
A point that's a minimum in some directions but a maximum in others—like a mountain pass or a horse saddle.
Lesson 95 — Local vs Global Optima
saliency map
A saliency map is a pixel-level visualization that highlights the spatial regions of an input image that had the greatest impact on a neural network's output.
Lesson 3231 — What Are Saliency Maps?
sample
Key Takeaway: A population is the complete set you want to understand; a sample is the smaller subset you actually observe.
Lesson 75 — Population vs Sample
Sample Efficiency
PPO is on-policy, requiring fresh experience for each update. In environments where data collection is expensive (real robots, expensive simulations), off-policy methods like SAC may be more practical.
Lesson 2314 — PPO in Practice: Success Stories and Limitations
Sample quality
A good schedule preserves image structure in early steps
Lesson 1526 — Variance Schedule: Controlling Noise Addition
Sample statistics
Sample statistics are the values we calculate from our sample data.
Lesson 75 — Population vs Sample
Sampling
You can generate new data by simply sampling z ~ N(0, I) and passing it through the decoder. Because the latent space is trained to match this distribution, random samples will decode into plausible outputs.
Lesson 1447 — Why the Prior Matters
sampling distribution
A sampling distribution is the probability distribution of these sample statistics (like the mean, variance, or standard deviation) across many possible samples.
Lesson 82 — Sampling Distributions
SARSA
SARSA is like learning from your actual driving experience, including all your cautious decisions and mistakes.
Lesson 2178 — Q-Learning vs SARSA: Key Differences
Saturation effects
Neural networks use activation functions like ReLU or sigmoid. When neurons saturate (flatten out), their gradients become zero or near-zero, even if those features are genuinely important. The gradient "dies" not because the feature doesn't matter, but because the neuron has hit its flat region.
Lesson 3234 — Why Raw Gradients Are Noisy
Scalability
The single-stack design scales efficiently as you add parameters
Lesson 1200 — Decoder-Only Design: Why GPT Diverged from BERT
scalar
A scalar (single number) is a 0-dimensional tensor
Lesson 775 — What is a Tensor?
Scale and coverage
A single research team can't test every edge case. Crowdsourcing naturally explores long-tail queries and niche domains.
Lesson 3177 — Chatbot Arena and Community Evaluation
Scale-independent evaluation
Scale-independent evaluation means you can compare models across different datasets or target ranges.
Lesson 473 — Mean Absolute Percentage Error (MAPE)
Scattered attention
Either the model is confused, or the task genuinely requires broad context integration.
Lesson 1059 — Understanding Attention Weight Visualization
Scheduling and triggers
Scheduling and triggers are the mechanisms that determine when your DAG executes.
Lesson 2864 — Scheduling and Triggers
Schema compliance
The JSON may be valid but not match your desired structure
Lesson 1913 — Native JSON Mode in Modern LLMs
score function
A score function is simply the gradient of the log-probability density with respect to the input data.
Lesson 1553 — Score Functions and the Score Matching Objective
score matching
At its core, score matching is about learning the score function—the gradient of the log probability of your data distribution.
Lesson 1535 — Connection to Score Matching
Score near -1
Point is probably in the wrong cluster (bad!)
Lesson 342 — Silhouette Score
Score near +1
Point is well-matched to its cluster and far from others (great!)
Lesson 342 — Silhouette Score
Score near 0
Point is on the border between clusters (ambiguous)
Lesson 342 — Silhouette Score
search space
The search space is the complete set of possible values you allow each hyperparameter to take when tuning your model.
Lesson 506 — The Hyperparameter Search Space
Seasonal decomposition
Seasonal decomposition is the process of separating that chord back into its individual notes: the long-term trend (where things are heading overall), the repeating seasonal pattern (predictable cycles like weekly or yearly fluctuations), and the residual or noise (random, unexplained variation).
Lesson 2403 — Seasonal Decomposition
Secondary models
A specialized model scores factual accuracy or safety
Lesson 1943 — External Validators in Refinement Loops
Section headers
The H1/H2/H3 hierarchy the chunk belongs to
Lesson 1993 — Metadata Enrichment
Secure aggregation
Secure aggregation is a cryptographic protocol that allows a central server to compute the sum of inputs from many users while mathematically guaranteeing that no individual input is revealed—even to the server itself.
Lesson 3368 — Secure Aggregation Protocol
secure enclave
TEEs provide an isolated execution space called a secure enclave where sensitive computations can run.
Lesson 3373 — Trusted Execution Environments
Security severity
Targeted attacks are often more dangerous. Making a spam filter classify one message as "not spam" is targeted; making it misclassify randomly is untargeted.
Lesson 3379 — Targeted vs Untargeted Attacks
Self-Adversarial Training
The network slightly modifies images to fool itself, then learns from those "attacks"
Lesson 965 — YOLOv4 and YOLOv5: Speed and Accuracy Advances
Self-attention
Self-attention is like understanding grammar and word relationships within the English text itself
Lesson 1078 — Cross-Attention vs. Self-Attention Heads
Semantic segmentation
Labels every pixel with its class ("this pixel is sky, that pixel is tree, this pixel is road")
Lesson 975 — What Is Semantic Segmentation
Semi-linear structure
The diffusion ODE has a particular mathematical form that allows efficient high-order approximations
Lesson 1602 — DPM-Solver and ODE Solvers
sensitive features
Protected attributes (also called sensitive features) are characteristics of individuals that should not—legally or ethically—influence certain decisions.
Lesson 3294 — Protected Attributes and Sensitive Features
Sentiment analysis
The end of a sentence often flips meaning ("I thought it was good... NOT!")
Lesson 1024 — Bidirectional LSTMs and GRUs
Separation by masking
The network learns to predict a multiplicative mask for each source. When you multiply the mixture spectrogram by the vocal mask, you get (mostly) just the vocals.
Lesson 2481 — Audio Source Separation
shape
The shape is a tuple showing the length along each dimension.
Lesson 151 — Array Shapes and Dimensions in ML
Shared Context
Shared Context refers to common knowledge all agents can access: the current task state, goals, constraints, and environmental observations.
Lesson 2120 — Shared Context and Memory in Multi-Agent Systems
Shared Memory
A small (48-164KB per streaming multiprocessor), ultra-fast scratchpad explicitly controlled by kernels. Custom CUDA kernels use shared memory to stage frequently-accessed data, reducing VRAM trips. This is critical for optimized operations like flash attention.
Lesson 2120 — Shared Context and Memory in Multi-Agent SystemsLesson 2935 — Understanding GPU Memory Hierarchy for Inference
Sharpening
A low temperature is applied to the teacher's softmax outputs (like we saw in contrastive learning), making the predictions more confident and peaked. The student, however, uses a higher temperature, creating softer targets.
Lesson 2567 — DINO: Self-Distillation with No Labels
Short sequences
The context vector may be sufficient
Lesson 1027 — Context Vector as Bottleneck
Short-term memory
Short-term memory (working memory) is the agent's current context—the immediate conversation, the task at hand, and recent observations from the environment.
Lesson 2097 — Short-Term vs Long-Term Memory in Agents
Short-term optimization
Short-term optimization means telling clients their form is perfect and they can skip hard exercises—instant satisfaction, five-star ratings.
Lesson 3445 — Short-Term vs Long-Term Alignment
SiLU
Unlike ReLU which abruptly cuts off negative values, Swish (also called SiLU - Sigmoid Linear Unit) creates a smooth, self-gated activation by multiplying the input by its own sigmoid.
Lesson 660 — Swish and SiLU: Self-Gated Activations
Simplicity
Absolute encodings are conceptually simple—each position has a fixed code. This makes implementation and debugging easier.
Lesson 1086 — Absolute Positional Embeddings: Advantages and Limitations
SimSiam
SimSiam is the most memory-efficient: no momentum encoder, no extra memory banks—just stop-gradient.
Lesson 2570 — Comparing Non-Contrastive Approaches
Single pass
Each point is visited exactly once
Lesson 349 — DBSCAN Algorithm Step-by-Step
skip-gram architecture
While you've learned that the distributional hypothesis tells us "words that appear in similar contexts have similar meanings," the skip-gram architecture is one specific way to learn those dense word embeddings.
Lesson 1119 — Word2Vec: Skip-gram Architecture
Slice-based evaluation
Slice-based evaluation means systematically measuring model performance on meaningful subsets (slices) of your data—defined by features, combinations of features, or other criteria—to uncover hidden disparities.
Lesson 3127 — What is Slice-Based Evaluation?
Slower convergence
The algorithm takes many more communication rounds to reach acceptable performance
Lesson 3356 — Handling Non-IID Data
small margin
A small margin means your decision boundary is close to some training points—it's confident but fragile
Lesson 268 — The Concept of Margin
Small negative values
Small negative values (close to zero) are usually statistical noise—treat them as unimportant features.
Lesson 3201 — Interpreting Negative Importance Values
Smooth evolution
The encoder evolves gradually, not abruptly
Lesson 2555 — Momentum Update Strategy
Smoothness
Unlike ReLU's sharp corner at zero, GELU is differentiable everywhere, which can improve gradient flow
Lesson 659 — GELU: Gaussian Error Linear Units
Soft targets
Soft targets are the full probability distribution output by the teacher model—capturing not just what the teacher predicts, but how confident it is and which alternative classes seemed plausible.
Lesson 2680 — Soft Targets and Temperature Scaling
Softmax loss on pairs
Classify whether sentence pairs are similar
Lesson 1972 — Sentence Transformers Architecture
Span
Span is the collection of all possible destinations you can reach using linear combinations (addition and scalar multiplication) of your vectors.
Lesson 10 — Linear Independence and Span
span prediction
This is called span prediction, and it works by predicting two things: where the answer starts and where it ends.
Lesson 1300 — Span Prediction with BERT
Span-based
Answers are always continuous sequences from the context
Lesson 1298 — Extractive QA Fundamentals
Sparse problems
Many machine learning problems have sparse solutions (most coefficients are zero), and coordinate descent can efficiently identify and update only the relevant variables
Lesson 109 — Coordinate Descent
Sparse Rewards
PPO struggles when feedback is rare and delayed. Without frequent reward signals, advantage estimates become unreliable, leading to slow or failed learning.
Lesson 2314 — PPO in Practice: Success Stories and Limitations
Sparsity
Weighted adjacency matrices remain sparse; computational efficiency is preserved
Lesson 2507 — Handling Directed and Weighted Graphs
sparsity constraint
The sparsity constraint is crucial—it forces the model to use only a few features at a time, preventing it from learning "cheating" distributed representations.
Lesson 3276 — Sparse Autoencoders for Disentanglement
sparsity target
Your sparsity target is simply the final percentage of weights you want to prune.
Lesson 2670 — Pruning Schedules and Sparsity Targets
Spectral envelope
The overall frequency distribution that identifies vowels and consonants
Lesson 2446 — Speech Signal Fundamentals
Spectral normalization
Spectral normalization is a technique that normalizes each weight matrix in your discriminator by dividing it by its spectral norm—the largest singular value of that matrix.
Lesson 1508 — Spectral Normalization
Speed
Greedy decoding is extremely fast—just one forward pass per token, no search overhead.
Lesson 336 — Naive Bayes Advantages and LimitationsLesson 1191 — Greedy Decoding
Speed improvements
The denoising network (U-Net) processes smaller tensors, meaning:
Lesson 1575 — Computational Benefits of Latent Diffusion
spherical
Think of it like buying furniture: full covariance is custom-made (expensive, perfect fit), diagonal is adjustable (width/height independent), spherical is one-size (cheap, fast), and tied means all rooms get identical furniture (consistent but less flexible).
Lesson 371 — Covariance Structure Constraints
Stable Learning
Low-resolution patterns are easier to learn first
Lesson 1485 — Progressive Growing of GANs (ProGAN)
Stage 2
Gradients + optimizer states are your bottleneck; balanced approach
Lesson 2748 — Memory vs Communication Tradeoffs
Staging
Model passed initial tests and is ready for validation in a production-like environment
Lesson 2832 — Model Staging and Promotion
state space
The state space is the set of all possible states the environment can be in.
Lesson 2134 — States, Actions, and State Spaces
States
Each cell in the grid is a state (e.g., position (2,3))
Lesson 2145 — Gridworld: A Classic MDP Example
Static quantization
Static quantization is like a standardized recipe where every measurement is predetermined, including seasonings—faster execution but requires testing batches first to get it right.
Lesson 2632 — Dynamic vs Static Quantization
Static shape handling
Static shape handling means your model is compiled and optimized assuming inputs always have the same dimensions—for example, images always 224×224 or sequences always length 512.
Lesson 2952 — Static vs Dynamic Shape Handling
Static thresholds
Static thresholds are simple but brittle: "Alert if error rate > 5%." They break when normal behavior changes (weekends vs weekdays, seasonal patterns).
Lesson 3023 — Alerting Strategies and Thresholds
Stationarity
Stationarity means your time series has constant statistical properties over time: specifically, constant mean, variance, and autocorrelation (how values relate to their own past).
Lesson 2386 — Stationarity and Why It Matters
Statistical power
Statistical power is critical (detecting small performance differences)
Lesson 3119 — Size vs Quality Tradeoffs
Statistical significance
The effect is unlikely due to chance (p-value)
Lesson 3078 — Interpreting A/B Test Results
Steps
Steps are individual optimizer updates (batches processed).
Lesson 1708 — Training Duration and Convergence
stochastic noise
This stochastic noise is similar to dropout—it prevents the network from relying too heavily on exact activation values and forces it to learn more robust features.
Lesson 764 — Normalization as Regularization
Stochastic regularization
The probabilistic weighting acts as implicit regularization
Lesson 659 — GELU: Gaussian Error Linear Units
Straightforward training
The model simply learns to predict the next token given all previous tokens
Lesson 1186 — Left-to-Right vs Bidirectional Context
Stratified K-Fold
Stratified K-Fold is a smarter version of K-Fold that preserves the class distribution in every fold.
Lesson 494 — Stratified K-Fold for Classification
Streaming support
Streaming support is gRPC's superpower: you can stream inputs for online learning scenarios, stream outputs for generated text/images, or both simultaneously — impossible with basic REST.
Lesson 2905 — gRPC for High-Performance Serving
Stride
Stride is the parameter that controls how far the filter shifts with each step—both horizontally and vertically.
Lesson 855 — Stride: Controlling Step Size
stride of 1
A stride of 1 means the filter moves one pixel at a time, computing an output value at nearly every possible position.
Lesson 855 — Stride: Controlling Step Size
stride of 2
A stride of 2 means the filter jumps two pixels with each step, effectively skipping positions.
Lesson 855 — Stride: Controlling Step Size
Structured vs Unstructured
Unstructured pruning (removing individual weights) offers flexibility but requires specialized hardware to achieve speedups. Structured pruning (removing entire channels or layers) is hardware-friendly but may harm accuracy more.
Lesson 2666 — Why Prune: Benefits and Trade-offs
Style and tone
The manner of response you prefer
Lesson 1832 — Introduction to Few-Shot Prompting
Subjectivity
Preferences often depend on subjective cultural context, personal values, or expertise. What one annotator considers "helpful" another might find "verbose." Aggregating these views is challenging.
Lesson 1817 — Limitations of Human Feedback and Motivation for RLAIF
Subpopulation disparities
A fraud detector might excel on common transaction types but fail on rare, high-value cases
Lesson 3128 — Why Aggregate Metrics Hide Problems
Success rate
Success rate is the primary metric: the percentage of inputs where the adversarial perturbation caused misclassification.
Lesson 3400 — Evaluating Attack Success and Perturbation Budgets
Successive Halving
Successive Halving is a smarter approach: start by training many configurations with a small budget (few iterations, small data subset), then progressively eliminate the worst performers and give more resources only to the promising ones.
Lesson 513 — Successive Halving and Early Stopping
Sufficient for many tasks
For most language and vision tasks, knowing "this is the 5th token" provides enough positional information for the model to learn meaningful patterns.
Lesson 1086 — Absolute Positional Embeddings: Advantages and Limitations
Sum
Sum is theoretically most powerful because it preserves neighbor count information.
Lesson 2503 — Aggregation Functions: Mean, Max, Sum
Summarization
The full document must be encoded before creating a condensed version
Lesson 1102 — Encoder-Decoder vs Decoder-Only Trade-offs
Summary version
A condensed, high-level distillation
Lesson 1995 — Multi-Representation Chunking
Superior accuracy
The model attends across both inputs, capturing nuanced relevance signals
Lesson 2006 — Bi-Encoder vs Cross-Encoder Trade-offs
Support Set
The few labeled examples (e.g., 5 images per class) used to adapt or condition your model for this specific task
Lesson 2585 — Support Set vs Query SetLesson 2606 — The Meta-Learning Problem Formulation
Swapping
Swapping is the gold standard: always evaluate each pair twice with reversed order, then aggregate results (e.g., count a win only if the model wins in both positions, or average the scores).
Lesson 3164 — Position Bias in LLM Judges
Synapses
Synapses are the connection points where signals pass between neurons.
Lesson 589 — The Biological Neuron: Inspiration for Artificial Networks
Synthetic request injection
Synthetic request injection is the core technique: before marking an instance "ready," send dummy inference requests through the pipeline.
Lesson 3009 — Model Warmup and Cold Start Optimization
System
Think of it like a play: the system is the director's notes (invisible to the audience but guiding everything), the user is the audience asking questions, and the assistant is the actor performing on stage.
Lesson 1232 — Instruction Format and Template DesignLesson 1752 — Instruction Format and TemplatesLesson 1854 — System vs User vs Assistant Messages
system prompt
A system prompt is a special instruction block sent to an LLM that sets persistent context, defines the model's persona, and establishes behavioral constraints—all before the user's actual question or task arrives.
Lesson 1853 — What Are System Prompts?

T

t-SNE
t-SNE is stochastic—run it twice, get different results (though patterns remain similar).
Lesson 403 — UMAP vs t-SNE: Comparative Analysis
Tabular data
Often the features are already interpretable (age, income), but may need discretization (converting continuous values into bins like "high income" vs "low income")
Lesson 3223 — Interpretable Representations
tags
Think of W&B's organizational system like a filing cabinet: projects are drawers, groups are folders within drawers, tags are colored sticky notes, and notes are detailed labels.
Lesson 2816 — W&B Run Management and Organization
Target
The appropriate helpful response
Lesson 1749 — What Is Instruction Tuning?
target network
A target network is a separate copy of your Q-network that generates the target values in your loss function.
Lesson 2211 — Target Networks for StabilityLesson 2223 — Target Network: Stabilizing Q-Learning
target statistics
CatBoost uses a clever technique called target statistics (also known as greedy target-based statistics).
Lesson 318 — CatBoost: Handling Categorical Features Natively
Task pattern
The transformation rule you want applied
Lesson 1832 — Introduction to Few-Shot Prompting
task-specific head
A task-specific head is just a small neural network (often a single linear layer) that you attach on top of BERT to map this [CLS] representation to your specific classification problem.
Lesson 1174 — Task-Specific Heads for Classification
task-specific system prompts
While generic system prompts define general behavior, task-specific system prompts are precision instruments tailored to excel at particular activities.
Lesson 1859 — Task-Specific System Prompts
Tasks
Tasks are the individual units of work, like loading data or training a model.
Lesson 2875 — Prefect Architecture and Task API
TD approach
After driving one block, estimate remaining time based on your current belief. Less noisy (one block is more predictable), but initially wrong if your beliefs are off.
Lesson 2173 — TD vs Monte Carlo: Bias-Variance Tradeoff
TD3
TD3 is also sample-efficient but may require more samples in sparse-reward environments where exploration is critical.
Lesson 2324 — SAC vs TD3: When to Use Which
Teacher forcing
Teacher forcing means using the ground truth token from the training data as the decoder's input at each time step, instead of the decoder's own prediction.
Lesson 1029 — Teacher Forcing in Training
Tecton
Tecton is an enterprise-grade, managed platform built by Feast's original creators.
Lesson 2890 — Feature Store Tools: Feast, Tecton, and Alternatives
temperature
In the InfoNCE loss you just learned, there's a small but mighty parameter called temperature (τ, tau) that sits in the denominator of the softmax.
Lesson 2538 — Temperature in Contrastive Loss
Temporal and geographic slicing
Temporal and geographic slicing means deliberately splitting your evaluation data by time windows and location attributes to expose these hidden weaknesses.
Lesson 3133 — Temporal and Geographic Slices
Temporal Modeling
Temporal Modeling is the heart of video understanding—learning which frames matter and how they relate sequentially.
Lesson 995 — Video Understanding Tasks
Temporal patterns
The rhythm and duration of sounds that distinguish phonemes (basic speech units like "p" vs "b")
Lesson 2446 — Speech Signal Fundamentals
tensor
A tensor is simply a multidimensional array—a container for numbers arranged in a grid with one or more dimensions.
Lesson 775 — What is a Tensor?
Tensor core usage
Specialized hardware for matrix operations is more energy-efficient per operation than standard CUDA cores
Lesson 3469 — GPU Power Consumption and Efficiency
TensorFlow Model Analysis
TensorFlow Model Analysis is the industry-standard library for slice-based evaluation.
Lesson 3136 — Tools and Workflows for Slice-Based Analysis
terminal state
In reinforcement learning, a terminal state is where an episode ends—the game is over, the agent reached the goal, or failed catastrophically.
Lesson 2217 — Handling Terminal States
Testable
You should be able to apply the principle to any model output and get a clear yes/no answer. "Choose the response that is more helpful" is testable; "be better" is not.
Lesson 1823 — Writing and Selecting Constitutional Principles
Text
Text is discrete, sequential, and symbolic
Lesson 1374 — Vision-Language Alignment Problem
text degeneration
This is called text degeneration or repetition.
Lesson 1323 — Repetition and Degeneration Problems
text encoder
A text encoder is a pretrained language model (like CLIP's text encoder or T5) that transforms your text prompt into a rich numerical embedding—a vector that captures the semantic meaning of your words.
Lesson 1590 — Text Encoder Integration
Think of it this way
Just as VAEs use ELBO to bridge exact likelihood (intractable) with a trainable bound, DDPMs use ELBO to connect the ideal reverse diffusion process (unknown) with your learned denoising network. The simplified loss is just the most practical form of this bound for training.
Lesson 305 — Bagging for Other Base LearnersLesson 1547 — Connection to Variational Lower Bound
Threat modeling
Threat modeling is the structured process of anticipating how your language model could be attacked, misused, or fail—before those problems emerge in production.
Lesson 3448 — Threat Modeling for Language Models
Threshold adjustment
Threshold adjustment means changing that cutoff point.
Lesson 545 — Threshold Adjustment for Imbalanced Data
Threshold optimization
Threshold optimization means setting different thresholds for different protected groups to satisfy fairness criteria.
Lesson 3312 — Threshold Optimization
Threshold-based secret sharing
Threshold-based secret sharing is the key.
Lesson 3371 — Dropout Resilience in Secure Aggregation
tied
Think of it like buying furniture: full covariance is custom-made (expensive, perfect fit), diagonal is adjustable (width/height independent), spherical is one-size (cheap, fast), and tied means all rooms get identical furniture (consistent but less flexible).
Lesson 371 — Covariance Structure Constraints
Tiered Decision Systems
Routine, low-risk cases are automated; medium-risk cases get human review; high-risk cases require multi-person approval.
Lesson 3491 — Human-in-the-Loop Design Patterns
Time
Training passes through all layers are slow, especially for models with billions of parameters
Lesson 1701 — What Full Fine-Tuning Means for LLMs
Time-varying covariates
Time-varying covariates are processed alongside the target sequence, often through separate pathways that merge with temporal representations
Lesson 2421 — Handling Covariates and External Features
Timeout or resource exhaustion
An action takes too long or hits limits
Lesson 2090 — Dynamic Replanning and Error Recovery
Tokenization
Tokenization is the process of breaking down raw text into smaller units called tokens—which could be words, subwords, or even individual characters—and mapping each token to a unique numerical identifier.
Lesson 1237 — What Is Tokenization and Why It Matters
Tool execution errors
A function returns an error code or exception
Lesson 2090 — Dynamic Replanning and Error Recovery
tool registry
An action space is the complete set of operations an agent can perform—its "toolbox." A tool registry is the system that catalogs these tools with structured descriptions so the agent knows what's available and how to use each one.
Lesson 2062 — Action Space and Tool Registry
tool schema
A tool schema is a structured specification that describes:
Lesson 2072 — Tool Schema Definition
top-k filtering
This is called top-k filtering or k-nearest neighbors (k-NN) selection.
Lesson 2361 — Neighborhood Selection and Top-K Filtering
Trade-off
Training hundreds of architectures across many generations is computationally expensive, though techniques like weight inheritance and early stopping help.
Lesson 2697 — Evolutionary Algorithms for NAS
Traditional security vulnerabilities
Traditional security vulnerabilities are the familiar weaknesses in software: SQL injection, buffer overflows, authentication bypass, insecure APIs, or exposed credentials.
Lesson 3522 — Security Vulnerabilities vs. AI-Specific Risks
Trained on controlled tasks
Synthetic data where ground truth is known (e.g., modular arithmetic, simple classification)
Lesson 3267 — Toy Models for Mechanistic Analysis
Training
Training is like building a factory: it's expensive, energy-intensive, and happens once (or infrequently when you retrain).
Lesson 947 — Intersection over Union (IoU)Lesson 3468 — Measuring ML Energy ConsumptionLesson 3471 — Training vs Inference Environmental Costs
Training instability
The agent might unlearn good behaviors due to noisy updates
Lesson 2255 — Variance in Policy Gradients
training-serving skew
This training-serving skew is one of the most insidious bugs in ML systems.
Lesson 2881 — What is a Feature Store and Why It Matters
Trajectory analysis
Trajectory analysis means examining the complete chain of reasoning steps, tool calls, observations, and actions the agent took—its "trajectory"—to understand the failure mode.
Lesson 2128 — Trajectory Analysis and Error Attribution
Trajectory Management
Trajectory Management means tracking the full reasoning chain.
Lesson 1908 — Implementing ReAct Agents
Transfer learning
Adapts parameters through gradient descent on the target task. The adaptation strategy (learning rate, which layers to fine-tune) is chosen manually.
Lesson 2607 — Meta-Learning vs Transfer Learning
transferability
This phenomenon is called transferability, and it's both fascinating and concerning for security.
Lesson 3381 — Transferability of Adversarial Examples
Transformation logic
The actual computation (e.g., aggregations, encodings, time-windowed calculations)
Lesson 2885 — Feature Definition and Registration
Translation and structured input-output tasks
Translation and structured input-output tasks are ideal for encoder-decoder models:
Lesson 1102 — Encoder-Decoder vs Decoder-Only Trade-offs
Translation invariance
The filter detects the same pattern regardless of where it appears in the input
Lesson 852 — Convolution as a Sliding Window
Transparency
Stakeholders should understand how models work, what data they use, and how decisions are made. This includes model cards, documentation, and explainability techniques—especially important when explaining limitations to non-technical users.
Lesson 3487 — Principles of Responsible AI DevelopmentLesson 3505 — Algorithmic Transparency and Explainability Requirements
tree
A tree is a connected graph with no cycles where every node (except the root) has exactly one parent.
Lesson 2488 — Common Graph Types: Trees, DAGs, and Bipartite Graphs
Trend detection
A 30-day moving average reveals medium-term trends better than daily noise
Lesson 2392 — Rolling Window Statistics
Truncation
Fast baseline when key info is at the start
Lesson 1178 — Handling Long Documents
Truncation Trick
At inference, BigGAN samples latent codes from a truncated normal distribution (cutting off extreme values). This trades diversity for quality—like choosing to stay closer to "safe" areas of the latent space where the model is most confident.
Lesson 1489 — BigGAN: Scaling Up GAN Training
trust region
A trust region is a constrained neighborhood around your current policy parameters where you "trust" your local linear or quadratic approximation of the objective function.
Lesson 1791 — The Trust Region ConstraintLesson 2291 — Trust Regions in Optimization

U

U-Net-style models
U-Net-style models are popular because they:
Lesson 2481 — Audio Source Separation
UMAP
UMAP is more deterministic with default settings, producing more consistent results across runs.
Lesson 403 — UMAP vs t-SNE: Comparative Analysis
underfitting
High bias (also called underfitting) occurs when your model is too simple to learn the underlying patterns in your data.
Lesson 521 — High Bias Diagnosis
Unified pretraining and generation
The same causal attention used during pretraining (next-token prediction) works seamlessly at inference
Lesson 1200 — Decoder-Only Design: Why GPT Diverged from BERT
union
The union is the total area covered by either circle.
Lesson 947 — Intersection over Union (IoU)
Units confusion
SHAP values are in the model's output units (log-odds for classifiers, not probabilities)
Lesson 3218 — SHAP in Practice: Implementation and Interpretation
Universal
A single patch can fool the model on many different images
Lesson 3385 — Adversarial Patches
Unsupervised
No labels at all. Use Isolation Forest, One-Class SVM, or statistical methods. Good when anomalies are truly unknown.
Lesson 380 — Anomaly Detection in Practice
Unweighted graphs
All edges are equal (you're either friends or not)
Lesson 2483 — What Is a Graph? Nodes, Edges, and Basic Terminology
update rule
The update rule is the formula that tells you exactly how to adjust your parameters after each step.
Lesson 213 — The Gradient Descent Update Rule
User
Think of it like a play: the system is the director's notes (invisible to the audience but guiding everything), the user is the audience asking questions, and the assistant is the actor performing on stage.
Lesson 1232 — Instruction Format and Template DesignLesson 1752 — Instruction Format and TemplatesLesson 1854 — System vs User vs Assistant Messages
User prompt
The actual question or task
Lesson 1853 — What Are System Prompts?
User-based
The clerk finds other customers with similar purchase histories to yours, then suggests books those customers bought that you haven't.
Lesson 2350 — User-Based vs Item-Based Approaches
Utilization rate
A GPU at 100% utilization drawing full power versus 50% utilization with proportionally less
Lesson 3469 — GPU Power Consumption and Efficiency

V

validation curve
A validation curve is a diagnostic plot that shows how both training and validation scores change as you vary a single hyperparameter.
Lesson 524 — Validation Curves for Hyperparameters
value
Think of it like searching a library: your query is "books about transformers," each book's key is its title and topic tags, and the value is the book's actual content.
Lesson 1051 — Query, Key, Value: The Three Vectors
Variance change
Data that was tightly clustered (std=5) is now highly variable (std=25)
Lesson 3053 — Statistical Summary Monitoring
vector
A vector (list of numbers) is a 1-dimensional tensor
Lesson 775 — What is a Tensor?
vector database
A vector database is a specialized storage system optimized for:
Lesson 1957 — What Is a Vector Database and Why RAG Needs It
Versioned Test Sets
The infrastructure maintains multiple test set versions (public validation sets for development, private test sets for final ranking). This prevents overfitting to public benchmarks while still allowing researchers to iterate.
Lesson 3125 — Leaderboards and Evaluation Infrastructure
Vertical lines
Certain words (like punctuation or important keywords) get attention from many positions—these are "hub" words.
Lesson 1059 — Understanding Attention Weight Visualization
Vertical scatter
Wide spread means the feature's impact varies greatly
Lesson 3213 — SHAP Summary Plots and Feature Importance
ViLT
Single-stream architecture where image and text tokens are concatenated early and processed jointly
Lesson 1412 — Transformer-Based VQA Models
Visual Genome
Visual Genome is a landmark dataset that revolutionized this field by providing unprecedented detail about images.
Lesson 1384 — Visual Genome and Large-Scale VL Datasets
Visual priming
Certain objects correlate strongly with specific answers (e.g., "tennis" often appears when rackets are visible).
Lesson 1413 — VQA Evaluation and Bias Challenges
Volume explosion
The "space" becomes so vast that data points are increasingly sparse
Lesson 1961 — The Curse of Dimensionality in Vector Search
Vᵀ
Vᵀ is the transpose of an n×n orthogonal matrix (second rotation)
Lesson 22 — Singular Value Decomposition (SVD): Concept

W

Weaviate
Weaviate is an open-source vector database you deploy yourself (or use their managed cloud).
Lesson 1966 — Vector Database Options: Pinecone, Weaviate, Qdrant
weight decay
That's why L2 regularization is also called weight decay—it makes weights shrink slightly with every training step, unless the original loss function strongly demands they stay large.
Lesson 734 — L2 Regularization (Weight Decay) Fundamentals
Weight Dropping
Weight Dropping is a related technique often used in recurrent networks, where specific weight matrices (like recurrent connections) have dropout applied to them consistently across time steps.
Lesson 747 — DropConnect and Weight Dropping
Weight sharing
The same filter scans the entire image, drastically reducing parameters
Lesson 889 — LeNet-5: The First Successful CNN
Weight-only quantization
Weight-only quantization is a selective approach where you convert model weights (the learned parameters) from 32-bit floating point to lower precision (typically 8-bit integers), but leave activations at full precision during inference.
Lesson 2633 — Weight-Only Quantization
Weighted aggregation
Weighted aggregation is the most common approach: assign importance weights to each metric and compute a composite score:
Lesson 3101 — Multi-Task and Multi-Objective Evaluation
Weights
Weights are static parameters that change slowly through gradient updates.
Lesson 2648 — QAT for Activations vs Weights
When unsure
The memory saved is often negligible compared to the risk of gradient errors
Lesson 786 — In-place Operations and Memory
who
Having risk management processes isn't enough—you need a governance structure that defines who is responsible for what, how decisions get made, and where accountability lies when things go wrong.
Lesson 3536 — Risk Governance Structures
Why it matters
A non-trivial null space means the transformation loses information. Multiple different inputs produce the same output (or zero), making the transformation non-invertible.
Lesson 12 — Column Space and Null Space
Why this works
The power law relationship continues beyond the compute-optimal point—just more slowly. You get better downstream task performance, stronger reasoning, and improved generalization.
Lesson 1630 — Post-Chinchilla Training Strategies
wide component
The wide component is a linear model that learns direct feature interactions—essentially memorizing co-occurrence patterns through cross-product transformations.
Lesson 2367 — Wide & Deep Networks for Recommendations
Win rate
Win rate is the simplest preference metric: given two model outputs for the same input, a human (or LLM judge) picks the better one.
Lesson 3173 — Introduction to Win Rate Metrics
Word embeddings
Word embeddings are dense, low-dimensional vectors (typically 50-300 dimensions) where similar words have similar vectors.
Lesson 1117 — Why Word Embeddings: From One-Hot to Dense Vectors
WordPiece
WordPiece is more selective—it merges pairs that maximize likelihood, creating a vocabulary that better reflects language patterns rather than raw frequency.
Lesson 1264 — Comparing Tokenization Algorithms
Worker count increases
More participants in the All-Reduce means more coordination complexity
Lesson 2711 — Communication Overhead and Bottlenecks
world size
The rank is a unique ID for each process (0, 1, 2, ...), the world size is the total number of processes, and a process group is the communication channel connecting them all.
Lesson 2794 — Distributed Process Groups and Ranks

X

X^T
X^T is the transpose of your feature matrix
Lesson 193 — The Closed-Form Solution (Normal Equation)

Z

Zero
Vectors are perpendicular (unrelated)
Lesson 3 — Dot Product and Vector Similarity
Zero-copy operations
Branches share underlying data objects; only changes are stored separately.
Lesson 2844 — LakeFS for Data Lake Versioning
Zero-Shot Chain-of-Thought
Zero-Shot Chain-of-Thought is remarkably simple: just append the phrase "Let's think step by step" (or similar variants) to your prompt.
Lesson 1864 — Zero-Shot Chain-of-Thought with 'Let's Think Step by Step'
Zero-shot forecasting
Zero-shot forecasting means you can feed your time series directly into a pre-trained model like TimeGPT and get predictions immediately—no task-specific training required.
Lesson 2425 — Zero-Shot Forecasting with Foundation Models
zero-shot learning
This is called zero-shot learning: the model performs a task with zero training examples for that specific task.
Lesson 1203 — GPT-2's Zero-Shot Task Transfer
Zero-shot QA
Zero-shot QA means giving the model a question with context and expecting an answer—no examples provided.
Lesson 1310 — QA with Large Language Models