← Back to AI Engineering

AI Engineering Glossary

Key terms from the AI Engineering course, linked to the lesson that introduces each one.

507 terms.

A

Abstraction layers
Abstraction layers are your primary defense.
Lesson 1124 — Vendor Lock-in and Migration Strategies
Accelerate
Accelerate is Hugging Face's library that abstracts away the complexity of distributed computing.
Lesson 1076 — Setting Up Multi-GPU with Accelerate
Accuracy
Quick overview when classes are balanced
Lesson 796 — Classification Task Metrics
action space
An action space is the catalog of all possible actions your AI agent can perform in response to its observations and reasoning.
Lesson 589 — Action Space and Tool Calling
Active Requests
The number of in-flight LLM calls at this moment. A sudden spike might indicate a retry storm or API issues.
Lesson 1258 — Real-Time Monitoring Dashboards
Adaptation
Adaptation means modifying them strategically:
Lesson 825 — Public Benchmarks and Adaptation
Adapter caching
Adapter caching means keeping recently-used or frequently-accessed adapters in GPU or CPU memory so they're immediately available when the next request arrives.
Lesson 1376 — Adapter Caching and Warm-Up
adapter registry
An adapter registry is like a library catalog system for your adapters.
Lesson 1370 — Adapter Registry and Management
adaptive cards
Unlike simpler webhook-based bots (Slack, Discord), Teams bots leverage a stateful conversation framework that handles complex multi-turn dialogues, persistent context, and rich interactive components called adaptive cards.
Lesson 1823 — Microsoft Teams Bot Framework
Adding noise
Adding noise means injecting small, random distortions into the results to make it mathematically impossible to infer private details about any single person.
Lesson 1537 — Adding Noise to Model Outputs
Agent
An individual team member with a specific role, goal, and backstory. Think of agents as specialists—a researcher, writer, or analyst—each with their own capabilities and personality defined in their system prompt.
Lesson 704 — CrewAI Framework Fundamentals
Agent memory
Agent memory is the component that allows an AI agent to store and recall information from previous interactions, observations, and decisions.
Lesson 595 — What Is Agent Memory?
agent registry
In multi-agent systems, an agent registry is that directory.
Lesson 676 — Agent Registry and Discovery
Agent state
The conversation history or context window at that moment
Lesson 660 — Tracing Tool Calls and Context
AI agent
An AI agent is an autonomous system that continuously perceives its environment, makes decisions based on reasoning, and takes actions to achieve specific goals—without needing step-by-step human instructions for every move.
Lesson 585 — What is an AI Agent?
AI alignment
AI alignment is the challenge of ensuring AI systems act according to human values, intentions, and preferences—not just the narrow metrics we measure.
Lesson 1587 — What is AI Alignment
AIF360
Fairlearn (Microsoft) and AIF360 (IBM) are the two most widely adopted fairness toolkits.
Lesson 1574 — Fairness Metrics Implementation and Tools
Allowlisting
Allowlisting means explicitly defining what's allowed and blocking everything else.
Lesson 1502 — Allowlisting Safe Libraries and APIs
Alpha
Alpha is a scaling factor that controls how strongly the adapter's updates influence the base model.
Lesson 1349 — LoRA Hyperparameters: Rank and Alpha
Analogy
Think of consent like a library card. Opt-in means you must sign up to borrow books. Opt-out means everyone gets a card automatically but can return it. Explicit is signing a form; implicit is walking into the library.
Lesson 611 — ReAct Planning PatternLesson 947 — Vector Database Integration PatternsLesson 1373 — Batching Across AdaptersLesson 1545 — Consent Models for AI Training Data
Analytics
Measuring speaking pace, detecting pauses, or highlighting keywords
Lesson 1688 — Timestamp and Word-Level Alignment
Annotator training and calibration
Annotator training and calibration is the systematic process of teaching raters exactly what to look for and keeping their judgments aligned with your standards and each other.
Lesson 843 — Annotator Training and CalibrationLesson 854 — Annotator Training and Calibration
Anonymization
Anonymization is the irreversible removal or transformation of identifying information.
Lesson 1525 — Anonymization vs Pseudonymization: Key Differences
Apache Airflow
Apache Airflow is the veteran—widely adopted for data pipelines and batch AI workflows.
Lesson 1797 — Orchestration Frameworks Overview
API key
An API key is like a special password that identifies your application to an external service.
Lesson 1473 — API Keys in AI Applications
API keys
API keys are simple shared secrets—like a master password to your service.
Lesson 1845 — API Key vs OAuth: When to Use Each
Append citations programmatically
If the answer is factually correct but uncited, inject citations yourself based on chunk relevance scores
Lesson 367 — Handling Missing or Hallucinated Citations
Arguments
The exact parameters passed (including types)
Lesson 660 — Tracing Tool Calls and Context
Arize
Arize is built for ML observability and drift detection.
Lesson 1282 — Comparing Arize and Helicone Use Cases
array
A list of items, like ["apple", "banana", "cherry"]
Lesson 556 — Parameter Types and Required vs Optional Fields
Assistant
Think of it like a play: the system message is the stage directions, the user is one character speaking, and the assistant is another character responding.
Lesson 91 — System, User, and Assistant Message Roles
Asynchronous execution
Asynchronous execution means initiating multiple tool calls at once and gathering results as they complete.
Lesson 592 — Synchronous vs Asynchronous Execution
Asynchronous processing
Asynchronous processing means you don't wait for one frame to finish completely before starting the next.
Lesson 1664 — Real-Time Video Processing Pipelines
Audience targeting
Audience targeting means explicitly telling the model who the intended reader is, so it adjusts its language, depth, and style accordingly.
Lesson 133 — Audience Targeting
Audit trails
Log where each piece of data is stored and processed (building on lesson 1523)
Lesson 1524 — Regional Data Residency and Compliance
auto
Normal chatbot interactions where context determines if tools are needed
Lesson 552 — Forcing and Disabling Function Calls
AutoClasses
AutoClasses are smart wrappers that automatically detect and load the correct model architecture for you.
Lesson 51 — Understanding AutoClasses
Automatic state management
The chain handles passing data between steps
Lesson 506 — Sequential Chains
Available Tools
The functions or capabilities the agent can use (from your function registry)
Lesson 629 — Setting Up the Initial State
Average Rating
For explicit thumbs-up/down or star ratings, compute means across time windows (daily, weekly). Track this by feature, user segment, or model version to spot performance differences.
Lesson 1401 — Aggregating and Analyzing Feedback

B

Backstories
Context that shapes the agent's behavior and expertise (e.g., "You are a senior data analyst with 10 years of experience in market research")
Lesson 705 — Defining Crews and Assigning Roles in CrewAI
Backward compatibility
Backward compatibility means newer feature pipelines can still serve older models by maintaining transformations for previous schema versions.
Lesson 790 — Schema Evolution and VersioningLesson 1002 — Backward Compatibility and DeprecationLesson 1603 — Version Control for Serialized ModelsLesson 1629 — Feature Versioning and Backward Compatibility
BakLLaVA
LLaVA (Large Language and Vision Assistant) and BakLLaVA are two leading open-source VLMs you can download and run locally for image understanding tasks like captioning, visual question answering, and multi-turn conversations about images.
Lesson 1726 — Open-Source VLMs: LLaVA and Bakllava
base model
A base model is trained on general data without targeting any specific task.
Lesson 45 — Model Variants and Checkpoints
Baseline metrics and regression suites
Baseline metrics and regression suites are your scientific control group—they let you measure real impact, not just hope for it.
Lesson 1422 — Evaluation Before and After Model Updates
Batch processing
Batch processing means grouping multiple documents together and sending them in a single API call or processing them together locally.
Lesson 59 — Batch Processing and DataLoadersLesson 220 — Batch Processing for EmbeddingsLesson 477 — Batch Processing Fundamentals
Batch search
Batch search means bundling multiple queries into a single request, allowing the system to optimize execution and reduce network overhead.
Lesson 271 — Batch Search and Query Optimization
Batch size
Batch size is the number of inputs you process simultaneously in a single forward pass through the model.
Lesson 64 — Batch Size and ThroughputLesson 1071 — Batch Size and Throughput Planning
batch window
A batch window is the maximum time to wait for requests to accumulate before processing.
Lesson 1611 — Batching Strategies for Throughput
BeautifulSoup
BeautifulSoup is a Python library that parses HTML and lets you navigate the document structure like a tree.
Lesson 460 — Web Content and HTML Extraction
Better reasoning
The LLM can focus purely on strategic thinking without worrying about tool execution
Lesson 610 — Plan-and-Execute Architecture
BF16
FP16 is common on NVIDIA GPUs, while BF16 (bfloat16) is newer and handles extreme values better, making it safer for many models without careful tuning.
Lesson 70 — Mixed Precision Inference
Bias in AI systems
Bias in AI systems refers to systematic errors or unfair outcomes that consistently affect specific groups in model predictions or outputs.
Lesson 1555 — What is Bias in AI Systems
Binary judgments
Binary judgments are yes/no or pass/fail decisions.
Lesson 812 — Binary vs Scalar Judgments
blackboard
In multi-agent systems, a blackboard is a centralized data structure (like a dictionary, database, or object store) where agents post observations, partial results, hypotheses, or task updates.
Lesson 697 — Blackboard Architecture for Shared State
blackboard architecture
A blackboard architecture is a formal pattern where:
Lesson 681 — Shared Memory and Blackboard Architectures
Blind spots
The judge may not recognize sophisticated reasoning it couldn't produce itself
Lesson 809 — Choosing the Judge Model
break-even point
The break-even point is where total API costs equal total self-hosting costs.
Lesson 122 — API vs Self-Hosted Break-Even Analysis
Buffering
Buffering means temporarily holding received tokens in memory before displaying them.
Lesson 113 — Buffering and Display Strategies
Built-in Observability
Every task execution is logged with inputs, outputs, duration, and errors. You can visualize your entire pipeline flow in Prefect's UI, making debugging much easier than scattered print statements.
Lesson 1799 — Prefect for LLM Pipelines

C

Cache Hit Rate
Your cache hit rate is the percentage of requests served from cache versus those requiring fresh API calls:
Lesson 961 — Monitoring Cache Hit RatesLesson 1166 — Measuring Cache Hit Rates and Parallel Gains
Cache invalidation
Cache invalidation is the active removal of specific cache entries when you know they're no longer valid—for example, when source documents are updated or user preferences change.
Lesson 1159 — Cache Invalidation and TTL Strategies
Cached Responses
For common queries, return previously cached results. A slightly stale answer is often better than no answer, especially in RAG systems where document content changes slowly.
Lesson 1794 — Fallback Strategies and Graceful Degradation
Caching
Caching means storing frequently accessed state in fast memory (like Redis or in-process memory) rather than hitting the database repeatedly.
Lesson 724 — Performance Optimization for State Access
Calibration
Calibration is closely related: it means that when your model says "70% confident," it should actually be right 70% of the time — and this should hold consistently across groups.
Lesson 1568 — Predictive Parity and Calibration
Calibration Sessions
Calibration Sessions are regular check-ins where:
Lesson 843 — Annotator Training and Calibration
canary deployment
A canary deployment is similar—you deploy to a small "canary" group first and watch for problems before full release.
Lesson 1864 — Gradual Rollouts and Canary Deployments
canary deployments
Shadow testing and canary deployments are two strategies that reduce risk:
Lesson 836 — Shadow Testing and Canary Deployments
canary release
A canary release is like sending a canary into a coal mine before miners enter—you test something risky on a small scale first.
Lesson 916 — Canary Releases and Progressive Rollouts
Capacity-based limits
Set hard caps (e.g., store only the 1000 most recent memories, or 500KB of vector embeddings). When limits are reached, remove the oldest or lowest-scoring entries.
Lesson 604 — Forgetting and Memory Pruning
Centralized log aggregation
Centralized log aggregation means routing all logs from every component to a single platform where you can search, filter, and analyze them together.
Lesson 1509 — Centralized Log Aggregation
chain
A chain is a sequence of operations that processes data step-by-step.
Lesson 505 — Chains: The Core Abstraction
Chain-of-Thought
Chain-of-Thought is about thinking out loud.
Lesson 181 — ReAct vs Chain-of-Thought Differences
Challenges
Sentences vary wildly in length—one might be 5 words, another 50. Some "sentences" in technical docs or legal text are massive. You need reliable sentence boundary detection (trickier than it sounds with abbreviations like "Dr." or "Inc.").
Lesson 338 — Sentence-Based Chunking
character encoding
This system is called character encoding.
Lesson 470 — Character Encoding and Unicode Handling
Chatbots and conversational interfaces
Chatbots and conversational interfaces are prime candidates.
Lesson 932 — When to Use Synchronous Patterns
checkpoint
A checkpoint is a saved snapshot of a model at a specific point in its training.
Lesson 45 — Model Variants and Checkpoints
Checkpointable state
The entire graph state can be serialized, enabling resumable workflows
Lesson 706 — LangGraph for Multi-Agent State Management
Checkpointing
Checkpointing is the practice of creating these snapshots at strategic points: before expensive operations, after successful tool calls, or at fixed intervals.
Lesson 485 — Progress Tracking and CheckpointingLesson 621 — State Serialization and CheckpointingLesson 1771 — Intermediate Result Storage and CheckpointingLesson 1804 — Checkpointing and Recovery Patterns
Chroma
Chroma is the simplest to start with—think of it as the "SQLite of vector databases." It's embedded by default (runs in your Python process), perfect for prototyping and small-scale applications.
Lesson 305 — Open Source Vector DB Landscape
Chunked Transfer Encoding
Chunked Transfer Encoding is an HTTP mechanism that lets your server send data in pieces (chunks) without declaring a Content-Length header beforehand.
Lesson 996 — Chunked Transfer Encoding
Circuit breakers
Circuit breakers are monitoring patterns that detect failures and stop sending traffic to a failing component.
Lesson 918 — Rollback Strategies and Circuit Breakers
Citation quality metrics
Citation quality metrics are standardized measurements that help you assess whether your system is attributing information correctly, covering all sources it should, and only citing relevant material.
Lesson 368 — Citation Quality Metrics
Class distribution
Monitor which categories are being predicted. Sudden changes might indicate new data patterns.
Lesson 1659 — Monitoring Vision Model Performance
Classification
Classification means bucketing errors into types:
Lesson 1792 — Error Detection and Classification
Classification tasks
Sentiment analysis or topic categorization are direct pattern matches
Lesson 171 — When CoT Helps vs When It Doesn't
Closing the loop
Closing the loop means demonstrating that their input mattered, which encourages continued engagement and builds trust.
Lesson 1405 — Closing the Loop with Users
Coherence
The bot needs to remember what the user just said to respond appropriately.
Lesson 735 — Conversation Context Fundamentals
collection
In Chroma, a collection is like a table in a traditional database—it's a named container that holds your vectors, documents, and associated metadata.
Lesson 307 — Chroma: Collections and MetadataLesson 313 — Milvus: Collections and Indexes
Color coding
Different span types (LLM calls, tool usage, chains) are visually distinct
Lesson 1264 — LangSmith Trace Visualization and Debugging
Combine signals
CTR + dwell time + completion is stronger than any single metric
Lesson 1391 — Signal Extraction from Implicit Feedback
Commercial use
Commercial use means anything that generates revenue or supports a business — including internal company tools.
Lesson 42 — Model Licensing and Usage Rights
Comparing prompt variations
Comparing prompt variations means running multiple prompt candidates against the same test suite and evaluating them with:
Lesson 1170 — Comparing Prompt Variations
Comprehensive coverage
A research agent + fact-checker + summarizer together cover more ground than any single agent
Lesson 690 — Parallel Agent Execution
Concept Drift
Concept Drift is the most subtle: the relationship between inputs and correct outputs changes.
Lesson 1243 — Understanding Distribution Drift in LLM Systems
Concurrent
Concurrent means you place all orders at once and collect them as they're ready—much faster.
Lesson 1162 — Async/Await and Concurrent API Calls
Conditional availability
Conditional availability means deciding which groups or individual functions to send to the LLM based on runtime conditions.
Lesson 563 — Function Grouping and Conditional Availability
Conditional Routing
Edges can be conditional functions that inspect state and determine the next node dynamically. For example, after a web search, route to "summarize" if results are good, or "refine_query" if they're insufficient.
Lesson 1800 — LangGraph for Agent Workflows
Conflicting constraints
A field marked both required and with default: null
Lesson 982 — Validation for Structured Output Requests
Connection Establishment
The initial TCP handshake to open a connection to the API server. This happens once per connection but can add 50-200ms depending on geographic distance.
Lesson 1140 — Network Latency and API Response Times
Constitutional AI
Anthropic's Claude takes a fundamentally different approach called Constitutional AI.
Lesson 86 — Anthropic Claude API: Constitutional AI Approach
container
A container is a running instance of an image.
Lesson 1092 — Docker Basics for AI Engineers
Context
The LLM still gets enough surrounding text to understand references, pronouns, and flow
Lesson 389 — Sentence Window Retrieval
Context Assembly
Confirm retrieved chunks are properly formatted and passed to the LLM with the right prompt template.
Lesson 893 — Testing Complete RAG Pipelines
Context compression on-the-fly
Context compression on-the-fly means processing retrieved documents after retrieval but before prompt injection to extract only the most relevant parts.
Lesson 359 — Context Compression On-the-Fly
Context matters
A short dwell time on a simple answer is good; on complex content it's bad
Lesson 1391 — Signal Extraction from Implicit Feedback
Context relevance instructions
Context relevance instructions are prompt directives that tell the LLM to actively filter and prioritize the context you've provided.
Lesson 355 — Context Relevance Instructions
Context Window Issues
Truncated responses, ignored instructions buried in long prompts, or confusion when context is too large.
Lesson 1296 — Analyzing Prompt-Response Pairs
Continuous ground truth updates
Continuous ground truth updates means establishing processes to regularly refresh your evaluation datasets so they stay aligned with your system's current challenges.
Lesson 828 — Continuous Ground Truth Updates
Continuous red-teaming
Continuous red-teaming means systematically analyzing production data to discover new vulnerabilities, then feeding those insights back into automated adversarial testing that runs regularly alongside model updates.
Lesson 1471 — Continuous Red-Teaming in Production
Control
Control means you manage everything yourself—choosing exact configurations, customizing behavior, and owning your infrastructure.
Lesson 24 — Control vs Convenience Trade-offs
Control blast radius
If something breaks, only a small percentage is affected
Lesson 878 — Progressive Rollouts and Feature Flags
Convenience
Convenience means letting someone else handle the complexity.
Lesson 24 — Control vs Convenience Trade-offs
Conversation context
Conversation context is the accumulated information from previous exchanges between a user and a chatbot—essentially, the "memory" of what's been discussed so far.
Lesson 735 — Conversation Context Fundamentals
Conversation state
Conversation State is structured, extracted information that represents what the system knows about the conversation.
Lesson 581 — Limiting Available Tools by ContextLesson 742 — Conversation State vs Message History
ConversationBufferMemory
ConversationBufferMemory is LangChain's basic memory component that stores the entire conversation history in a simple buffer (like a list).
Lesson 509 — Memory: ConversationBufferMemory
correlation ID
A correlation ID (or trace ID) is a unique identifier—typically a UUID—generated at the start of each user request and passed along to every downstream operation.
Lesson 1222 — Correlation IDs and Request Tracing
Cost attribution by feature
Cost attribution by feature means labeling each API request with metadata that identifies which part of your application generated it.
Lesson 1179 — Cost Attribution by Feature
Cost gates
Cost gates are automated checks that enforce spending limits before tests run or deployments proceed.
Lesson 908 — Cost Gates and Budget Limits
Cost optimization
If query is simple → use small model, else → use large model
Lesson 1768 — Branching Logic and Conditional Steps
cost per interaction
Understanding your cost per interaction is essential to determining whether your AI product is financially viable at scale.
Lesson 1854 — Cost per Interaction and Unit Economics
Crew
In CrewAI, a crew is a coordinated team of agents working together toward a common objective.
Lesson 704 — CrewAI Framework FundamentalsLesson 705 — Defining Crews and Assigning Roles in CrewAI
Cron Schedules
Cron Schedules are time-based triggers that run pipelines at fixed intervals—daily at 2 AM, every Monday, hourly during business hours.
Lesson 495 — Scheduling and Triggering Strategies
Custom formats
Storing data in a provider-specific vector database schema
Lesson 22 — Evaluating Vendor Lock-in Risk

D

Data drift
The input distributions shift. Users start asking questions in new formats, using different vocabulary, or approaching your system with novel query patterns.
Lesson 1426 — Detecting and Addressing Model Degradation
Data pipeline infrastructure
Data pipeline infrastructure is the plumbing that collects all this chaos and delivers it in a usable form.
Lesson 16 — Data Pipeline Infrastructure
Data Transfer
The actual time to send your prompt and receive the response over the wire. Large prompts or slow connections significantly impact this. Network bandwidth and latency both matter here.
Lesson 1140 — Network Latency and API Response Times
Data versioning
Data versioning is like Git for your datasets—it captures snapshots of your training data at specific points in time.
Lesson 1322 — Data Versioning and Lineage
Delimiters
Delimiters are special characters or strings that mark boundaries in the output.
Lesson 158 — Delimiters and Markers for Parsing
deployment pipeline
A deployment pipeline is an automated sequence of stages that code, prompts, or models pass through before reaching users—think of it as a conveyor belt with quality checkpoints.
Lesson 920 — Deployment Pipelines and Approval Gates
Deployments
Pods are individual cooking stations, Deployments are the head chef's recipe and staffing plan, and Services are the waiters connecting customers to the kitchen.
Lesson 1102 — Kubernetes Core Concepts: Pods, Deployments, Services
deprecation
Meanwhile, deprecation is the gentle, transparent process of saying: "Version 1 still works, but please migrate to Version 2—we'll remove v1 in 6 months."
Lesson 1002 — Backward Compatibility and Deprecation
description
A plain-English explanation of what the function does and when to use it. This is critical—the LLM reads this to decide whether to call your function. Be specific: "Gets current weather for a given city" is better than "Weather function."
Lesson 555 — Function Schema Structure and OpenAI Format
Detection
Detection means instrumenting each step to catch failures.
Lesson 1792 — Error Detection and Classification
Deterministic testing
The same input produces the same behavior
Lesson 1301 — Reproducing Issues Locally
Device mapping
Device mapping is the strategy you use to decide which layers live on which GPU (or CPU) to balance memory usage and maximize throughput.
Lesson 1077 — Device Mapping Strategies
Direct Messages
Direct Messages are private conversations users initiate with your bot.
Lesson 1821 — Slack Event Handling and Commands
Discovery
Discovery is the process by which an agent queries this registry to find the right specialist for a task.
Lesson 676 — Agent Registry and Discovery
Distribution
Distribution means assigning those subtasks to agents based on their specific capabilities and roles.
Lesson 694 — Task Decomposition and Distribution
Distribution shift
The underlying relationship between inputs and outputs changes. What used to be a good answer six months ago may no longer satisfy current user expectations or domain requirements.
Lesson 1426 — Detecting and Addressing Model Degradation
Diversity-aware retrieval
Diversity-aware retrieval means going beyond pure similarity scoring.
Lesson 1580 — Retrieval Debiasing in RAG Systems
Document
A Document is the raw, unprocessed unit of data you feed into LlamaIndex.
Lesson 514 — Documents and Nodes: LlamaIndex Data Model
document loader
A document loader is a standardized interface that converts various file types (PDFs, Word docs, web pages, etc.) into a common format your AI pipeline can process.
Lesson 465 — Document Loaders and Abstractions
Documentation
Documentation is how you preserve what you've learned.
Lesson 1173 — Iteration Velocity and Documentation
Domain vocabulary
The specific jargon, abbreviations, and context your users bring
Lesson 1387 — The Production Data Advantage
Dropped Frames
The count of frames skipped or discarded. High drop rates indicate your pipeline can't keep pace with the input stream.
Lesson 1670 — Video Inference Monitoring and Debugging
Dynamic adapter loading
Dynamic adapter loading means loading adapter weights into memory only when a request requires them, then optionally unloading them to free space for the next adapter.
Lesson 1371 — Dynamic Adapter Loading

E

Early stopping
Early stopping means halting training when validation performance stops improving, even if training loss could go lower.
Lesson 1331 — Overfitting Detection and Early Stopping
Edge cases that matter
The weird, ambiguous, or poorly-formed inputs that happen in practice
Lesson 1387 — The Production Data Advantage
Edge computing
Edge computing means running CV models directly on devices near where data is captured—security cameras, drones, smartphones, IoT sensors—rather than sending data to remote cloud servers.
Lesson 1671 — Edge Computing Fundamentals for CV
Empty Citation Check
If your retrieved context is non-empty but the response contains zero citations, flag this as a potential issue.
Lesson 367 — Handling Missing or Hallucinated Citations
Endpointing
Endpointing is the process of determining when a speaker has completed their utterance and it's time for the system to respond.
Lesson 1708 — Endpointing and Turn-Taking Detection
Engineering time
Engineering time is typically the hidden giant.
Lesson 1083 — Understanding Total Cost of Ownership for Self-Hosted LLMs
Enum enforcement
Restricted choices are guaranteed
Lesson 760 — Function Calling for Structured Output
Equal Opportunity
Equal Opportunity means the model's true positive rate (recall) should be the same for all protected groups.
Lesson 1567 — Equal Opportunity and Equalized Odds
Equalized Odds
Also ensures unqualified applicants are rejected at similar rates across groups
Lesson 1567 — Equal Opportunity and Equalized Odds
error state
An error state is a named state in your machine specifically for handling failures.
Lesson 1784 — Error States and Recovery Strategies
Error-free parsing
The API won't return malformed JSON
Lesson 760 — Function Calling for Structured Output
Escaping
Escaping means converting special characters into safe representations.
Lesson 154 — Escaping and Sanitizing User Input
Evaluation and testing frameworks
Evaluation and testing frameworks are specialized tools designed to assess:
Lesson 17 — Evaluation and Testing Frameworks
Exact unlearning
Exact unlearning means retraining your model from scratch, excluding the requested data entirely.
Lesson 1549 — Exact Unlearning vs Approximate Unlearning
Example selection and pruning
Example selection and pruning means strategically choosing a smaller set of high-quality, diverse examples that teach the pattern without wasting context window space.
Lesson 1149 — Example Selection and Pruning
Explicit consent
Explicit consent is clear, affirmative action: a user clicks "I agree to have my data used for AI training." It's unambiguous and audit-friendly.
Lesson 1545 — Consent Models for AI Training Data
Explicit feedback
Explicit feedback is direct and intentional—users actively tell you what they think.
Lesson 1397 — Implicit vs Explicit Feedback
Explicit goal markers
The agent declares "task complete" in its output
Lesson 623 — Stopping Conditions: Goal Achievement
Explicit state representation
The current node shows exactly which agent is active
Lesson 706 — LangGraph for Multi-Agent State Management
Exponential Backoff
Exponential backoff means waiting progressively longer between retry attempts: first retry after 1 second, then 2 seconds, then 4, then 8, and so on.
Lesson 494 — Retry Logic and Error HandlingLesson 1793 — Retry Logic and Exponential Backoff
Extraction
Extraction means pulling out the individual steps from the model's response.
Lesson 172 — Extracting and Validating Reasoning Steps

F

Fail fast and learn quickly
A bad experiment costs minutes and cents, not days and dollars
Lesson 1386 — PEFT for Rapid Iteration and Experimentation
Fallback Mechanisms
Fallback Mechanisms are your safety nets:
Lesson 723 — State Recovery and Error Handling
Fast-path optimization
the first tier must be genuinely fast, or latency compounds
Lesson 1200 — Cascade Pattern for Model Routing
Feature drift
Feature drift is often the culprit: the statistical properties of your input features have changed, but your model still expects the old patterns.
Lesson 1628 — Feature Monitoring and Drift Detection
Feature flags
Feature flags are the control mechanism that lets you dynamically adjust these percentages without redeploying code.
Lesson 878 — Progressive Rollouts and Feature Flags
feature store
A feature store is a centralized repository that:
Lesson 1620 — Feature Store Fundamentals
feature store integration pattern
The feature store integration pattern is your blueprint for connecting both worlds to the same source of truth—ensuring the feature named user_30d_avg_purchase means exactly the same thing whether it's powering a training job or a real-time prediction.
Lesson 1635 — Feature Store Integration Patterns
Fine-tuning
Fine-tuning means training the model on your specific dataset to internalize patterns and behaviors.
Lesson 1303 — Fine-Tuning vs Prompt Engineering Trade-offs
Finite State Machine
A Finite State Machine consists of four fundamental elements that work together to model behavior:
Lesson 1778 — Finite State Machines (FSM) Basics
Fix output length and parameters
Set a low, fixed max_tokens and temperature=0 so generation time is deterministic and consistent.
Lesson 537 — Performance Comparison: Framework vs Raw
Fixed iteration count
Unlike text generation where sequences finish at different times, diffusion steps are predictable
Lesson 1028 — Batching for Different Model Architectures
Fixed-size chunking
Fixed-size chunking is the simplest strategy: you divide text into uniform segments of N characters or tokens, optionally with overlap between consecutive chunks.
Lesson 336 — Fixed-Size Chunking
fixture
In traditional software, a fixture is reusable setup code or data that multiple tests share.
Lesson 890 — Test Coverage and Fixtures for AI Systems
Flagging
Flagging is safer when you're uncertain—route borderline cases to human reviewers rather than auto-correcting and potentially changing intended meaning.
Lesson 1585 — Output Filtering and Rewriting
Flash Attention
Flash Attention is a hardware-aware algorithm that restructures attention computation to minimize memory reads/writes.
Lesson 1036 — Flash Attention and Kernel Optimizations
Flexibility
Deploy to environments where PyTorch is too heavy
Lesson 67 — ONNX Runtime Basics
Flows
Flows are the top-level containers for your pipeline logic—think of them as the "job" you want to run (like "update vector database" or "batch embed documents").
Lesson 491 — Prefect for Modern AI Workflows
FP16
FP16 is common on NVIDIA GPUs, while BF16 (bfloat16) is newer and handles extreme values better, making it safer for many models without careful tuning.
Lesson 70 — Mixed Precision Inference
Framework Benefits
These integrations eliminate boilerplate. You define the schema once in Pydantic, and the framework ensures the LLM's output conforms to it. Validation errors are caught automatically, and you receive type-safe Python objects.
Lesson 776 — Integration with LLM Frameworks
Function Calling
Function Calling (covered earlier) is more direct.
Lesson 648 — Comparing ReAct to Other Agent Patterns
Function grouping
Function grouping means organizing related functions together (e.g., "database_tools", "email_tools", "admin_tools").
Lesson 563 — Function Grouping and Conditional Availability

G

Gemini
Gemini is Google's newest multimodal model family (text, images, video, audio).
Lesson 1119 — Google Vertex AI Foundation Models
General principle
The tokenizer breaks text into the same pieces the model will see
Lesson 118 — Token Counting and Cost Estimation
Gold standards
Gold standards are questions or tasks where you already know the correct answer.
Lesson 845 — Quality Control and Gold Standards
golden dataset
A golden dataset is your curated, trusted collection of test cases with known correct outputs.
Lesson 824 — Golden Datasets and Versioning
GPU Auto-Scaling
Monitor queue depth and spin up/down GPU instances dynamically. Image generation has perfect "embarrassingly parallel" workloads—each request is independent.
Lesson 1744 — Production Image Generation Pipelines
GPU utilization
Percentage of GPU compute actually being used. Low utilization (<50%) suggests CPU bottlenecks or inefficient batching. Maxed-out GPU (>95%) means you're compute-bound.
Lesson 1659 — Monitoring Vision Model PerformanceLesson 1670 — Video Inference Monitoring and Debugging
Graceful Degradation
The main workflow continues while problems are queued
Lesson 723 — State Recovery and Error HandlingLesson 1796 — Dead Letter Queues and Manual Investigation
Ground truth
Ground truth is a collection of examples where you already know the correct answer.
Lesson 819 — What is Ground Truth and Why It Matters
guard
In a state machine, a guard is a boolean condition that must be satisfied before a transition fires.
Lesson 1782 — Guards and Conditional Transitions
Guardrail metrics
Guardrail metrics are your safety net—predefined thresholds that trigger automatic experiment termination when violated.
Lesson 876 — Guardrail Metrics and Early Stopping

H

Hallucinations
The model confidently stating false information or inventing details that weren't in your context. Look for citations to non-existent sources or facts that contradict your retrieved documents.
Lesson 1296 — Analyzing Prompt-Response Pairs
handler
The handler is where your preprocessing pipeline lives.
Lesson 1650 — TorchServe for Vision Models
Held-out test sets
Held-out test sets are your first line of defense.
Lesson 243 — Evaluating Fine-tuned Embeddings
Helicone
Helicone is a proxy-based logging platform.
Lesson 1282 — Comparing Arize and Helicone Use Cases
hierarchical organization
In multi-agent systems, hierarchical organization means arranging agents in layers, similar to a corporate org chart.
Lesson 691 — Hierarchical Agent Organization
HLS and DASH
HLS and DASH are adaptive streaming protocols—better for recorded content than live interaction due to 2-10 second latencies, but useful when you need broad device compatibility.
Lesson 1669 — WebRTC and Low-Latency Streaming Protocols
HMAC Signature Verification
HMAC Signature Verification is your primary defense.
Lesson 1831 — Webhook Security and Signature Verification
How to fix it
the expected format, range, or valid options
Lesson 578 — Error Messages for LLMs
Human-in-the-Loop Evaluation
Human-in-the-Loop Evaluation means involving real people to review your agent's decisions, tool selections, and reasoning chains—especially for complex or high-stakes tasks.
Lesson 667 — Human-in-the-Loop Evaluation

I

Idle resources
Idle resources are any cloud assets consuming money without providing value: stopped instances still attached to storage, orphaned disk volumes from deleted VMs, elastic IPs without attached instances, or load balancers pointing to nothing.
Lesson 1217 — Idle Resource Detection and Cleanup
If the guard returns false
transition is blocked; the system stays in the current state
Lesson 1782 — Guards and Conditional Transitions
image
An image is a blueprint—a read-only template containing your Python environment, model weights, dependencies, and code.
Lesson 1092 — Docker Basics for AI Engineers
Immediately after successful task completion
Immediately after successful task completion is your golden window.
Lesson 1399 — Timing and Context for Feedback Requests
Implicit feedback
Implicit feedback is indirect and behavioral—you infer quality from what users do, not what they say.
Lesson 1397 — Implicit vs Explicit Feedback
Impractical at scale
Handling thousands of deletion requests individually is impossible
Lesson 1548 — Machine Unlearning Fundamentals
In-memory state storage
In-memory state storage means keeping this information in your application's RAM using simple data structures like Python dictionaries.
Lesson 716 — In-Memory State Storage
Include full state
A complete checkpoint contains model weights, optimizer state, scheduler state, current epoch/step number, and training configuration. This ensures perfect resumability.
Lesson 1329 — Checkpoint Management and Recovery
Incomplete Responses
Answer is technically correct but unhelpful
Lesson 1872 — Identifying Failure Modes Through User Feedback
Inconsistency
The judge may lack the nuance to distinguish between subtle quality differences
Lesson 809 — Choosing the Judge Model
Indexes
Indexes are the top-level containers in Pinecone where you store and query vectors.
Lesson 296 — Pinecone Architecture and Concepts
Inference costs
Inference costs are typically your largest expense: LLM API calls (GPT-4, Claude, etc.), embedding generation, and model hosting.
Lesson 1880 — Cost Structure Analysis and Margin Calculation
Inference optimization
Inference optimization is the practice of making your model's predictions (inferences) faster, more efficient, and more cost-effective when serving real users.
Lesson 61 — What is Inference Optimization
Information Extraction
Beyond narrative descriptions, you can extract structured data: count objects, identify text within images (OCR-like tasks), determine spatial relationships ("the cat is on the table"), or answer specific questions about image content.
Lesson 1739 — Image Understanding and Captioning
input
The text (or list of texts) you want to embed
Lesson 215 — OpenAI Embeddings API Overview
Input and output snapshots
The raw user input and generated response (respecting privacy requirements)
Lesson 1462 — Logging and Audit Trails
Input examples
The prompts or queries your system receives
Lesson 1265 — Creating and Managing Datasets in LangSmith
Inputs and outputs
The exact text or data that went in and came out
Lesson 1264 — LangSmith Trace Visualization and Debugging
intents
Unlike Slack's event-driven architecture, Discord bots connect via a persistent WebSocket gateway and use a more granular permissions system called intents.
Lesson 1822 — Discord Bot Development with LLMs
Intersectional fairness
Intersectional fairness means examining AI system performance across combinations of protected attributes simultaneously, not just in isolation.
Lesson 1573 — Intersectionality and Multi-attribute Fairness
Iteration velocity
Iteration velocity is how quickly you can test new ideas.
Lesson 1173 — Iteration Velocity and Documentation
Iterative Prompt Refinement
Iterative Prompt Refinement is the practice of treating prompt engineering like debugging code.
Lesson 136 — Iterative Prompt Refinement
Iterative Refinement
Review disagreements, update guidelines, retrain annotators, and repeat. The goal is consistent, reliable labeling.
Lesson 821 — Manual Annotation Workflows

J

Joblib
Joblib is built specifically for this use case.
Lesson 1599 — Joblib for Efficient Persistence
JSON mode
JSON mode is a setting available in modern LLM APIs (like OpenAI's GPT-4, Anthropic's Claude, and others) that guarantees the model's response will be valid JSON.
Lesson 756 — JSON Mode Basics

K

Key rotation
Key rotation means cycling through a pool of API keys automatically.
Lesson 103 — Multi-Key Rotation Strategies
knowledge base
In RAG, the knowledge base is the collection of information your system searches through to find relevant context for the LLM.
Lesson 329 — The Knowledge Base in RAG
knowledge graph
This is called a knowledge graph—a network of nodes (entities) connected by edges (relationships).
Lesson 601 — Entity Memory and Knowledge Graphs

L

Lagging indicators
Lagging indicators are outcome-based measurements you can only collect after something has happened—like monthly user retention, total revenue, or final NPS scores.
Lesson 1857 — Leading vs Lagging Indicators
Language Support
Whisper is your Swiss Army knife for multilingual scenarios. Wav2Vec2 has strong coverage for major languages but may lack models for low-resource languages. Conformer availability depends on the specific implementation and training data.
Lesson 1713 — ASR Model Landscape and Selection Criteria
Latency
Smaller Wav2Vec2 models often have the edge for real-time applications. Whisper's "tiny" and "base" variants balance speed and accuracy. Conformer models vary widely—some are optimized for streaming with low latency, others prioritize batch accuracy.
Lesson 62 — Measuring Inference PerformanceLesson 262 — Recall vs Latency ConfigurationLesson 411 — Latency and Throughput MetricsLesson 1019 — Batch Size SelectionLesson 1636 — Hybrid Architectures and PrecomputationLesson 1713 — ASR Model Landscape and Selection Criteria
Latency Targets
If requests are taking too long, shrink batches to reduce queueing delay—even if it means lower throughput.
Lesson 1025 — Adaptive Batching Strategies
Latent failures
An LLM might slowly drift in quality over days as user queries change, your cached contexts become stale, or model behavior shifts. Traditional alerts won't catch this gradual degradation.
Lesson 1219 — Why Observability Matters for LLM Systems
Least privilege
Least privilege means granting only the minimum permissions needed for each role to perform their job—nothing more.
Lesson 1521 — Access Controls and Role-Based PermissionsLesson 1843 — Scoped Permissions and Least Privilege
Length and verbosity control
Length and verbosity control means explicitly telling the model how much to say: a single sentence, exactly 100 words, three bullet points, or a comprehensive essay.
Lesson 132 — Length and Verbosity Control
Limits
Limits are the maximum resources your pod can consume—like the fire code capacity of that room.
Lesson 1105 — Resource Requests and Limits for GPU Workloads
LlamaIndex
LangChain and LlamaIndex are two popular examples:
Lesson 13 — Orchestration Frameworks Overview
LLM-based relevance scoring
LLM-based relevance scoring means prompting a language model to evaluate whether a retrieved document answers or relates to a given query.
Lesson 410 — LLM-Based Relevance Scoring
LocalAI
LocalAI is that knife—a drop-in replacement for OpenAI's API that runs locally and handles text generation, embeddings, image generation, audio transcription, and more, all through familiar endpoints.
Lesson 1055 — LocalAI: Multi-Model Local Serving
Logit biasing
Logit biasing means adjusting these scores before selection, making certain tokens more or less likely.
Lesson 779 — Logit Biasing and Token Masking
Long-running tasks
A document processing pipeline with OCR, embedding, and summarization can run for hours without losing progress
Lesson 1798 — Temporal for AI Workflows
Long-term memory integration
Long-term memory integration means connecting your chatbot to persistent storage systems like vector databases or knowledge bases so it can recall past interactions, user preferences, and learned facts across multiple sessions.
Lesson 744 — Long-Term Memory Integration
loss function
Your loss function is your model's north star—it optimizes ruthlessly for whatever you measure.
Lesson 1557 — Sources of Bias: Model Architecture and Objectives
Low value
The scenario is unrealistic or extremely rare
Lesson 838 — Maintaining and Evolving Your Regression Suite
Lower latency
Skip silence means faster end-to-end transcription
Lesson 1706 — Voice Activity Detection (VAD) in Real-Time

M

Majority voting
The simplest approach. If most agents agree on an answer, that becomes the final result. Works best when outputs are discrete (classifications, yes/no decisions).
Lesson 189 — Majority Voting and Answer AggregationLesson 695 — Result Aggregation Strategies
malicious intent classifier
A malicious intent classifier is a dedicated ML model—often a simpler, faster model than your main LLM—trained specifically to recognize patterns associated with adversarial behavior.
Lesson 1489 — Detecting Malicious Intent with Classifiers
Managed Endpoints
Managed Endpoints are the key deployment mechanism.
Lesson 1117 — Azure Machine Learning for Custom Models
Maximal Marginal Relevance
Maximal Marginal Relevance is a re-ranking technique that balances two competing goals:
Lesson 273 — Diversity and MMR in Search Results
maximum iteration limit
Setting a maximum iteration limit is your safety valve.
Lesson 622 — Stopping Conditions: Max Iterations
Memory footprint
A 7B model at FP16 needs ~14GB. Two such models need 28GB+ (plus KV cache overhead). Check if they fit simultaneously or require swapping.
Lesson 1070 — Multi-Model Serving Considerations
Message History
Message History is the raw transcript—a chronological log of every message exchanged between user and assistant.
Lesson 742 — Conversation State vs Message History
Message History Formats
This is why Conversation State vs Message History (lesson 742) and proper Message History Formats (lesson 736) are foundational—they give the model the raw material needed for resolution.
Lesson 743 — Reference Resolution Across Turns
Message passing
Message passing is the mechanism that enables this communication.
Lesson 679 — Message Passing Between Agents
Metadata
Metadata are key-value pairs attached to each vector that describe non-semantic attributes (like {"author": "Alice", "year": 2023}).
Lesson 307 — Chroma: Collections and Metadata
Metrics
Instrument your service with Prometheus-compatible metrics. Track request rate, error rate, latency percentiles (p50, p95, p99), GPU utilization, and queue depth. These numbers tell you what's actually happening versus what you think is happening.
Lesson 1016 — Production Deployment Checklist
Milvus
Milvus is the enterprise-grade heavyweight, built for billion-scale vector search.
Lesson 305 — Open Source Vector DB Landscape
Minimal complexity
Your system is simple enough that a framework adds unnecessary weight
Lesson 712 — Framework Selection and Custom Solutions
Minimizing database queries
Minimizing database queries means batching operations and avoiding redundant lookups.
Lesson 724 — Performance Optimization for State Access
Misaligned objectives
The model optimizes for measured alignment metrics rather than true human values
Lesson 1596 — Alignment Tradeoffs and Failure Modes
Missing Required Fields
A request arrives without a prompt or model_name that your endpoint expects.
Lesson 976 — Handling Missing and Invalid Parameters
Mixed precision
Mixed precision means using less precise formats:
Lesson 70 — Mixed Precision Inference
MLflow Model Registry
MLflow Model Registry is the industry standard—integrate model logging in training, then promote versions via UI or API.
Lesson 1610 — Model Registry and Version Management
model
The embedding model version (e.g., text-embedding-3-small, text-embedding-3-large, text-embedding-ada-002)
Lesson 215 — OpenAI Embeddings API Overview
Model artifacts
While models themselves can be read-only and easily replicated (just copy files), metadata about which model version is active requires coordination.
Lesson 911 — Model Versioning FundamentalsLesson 1131 — Data Replication for Multi-Region SystemsLesson 1338 — Model Registry and Version Management
Model capability gaps
Model capability gaps are fundamental limitations in what a model can do—like asking a small language model to perform complex multi-step reasoning, or expecting a text-only model to understand images.
Lesson 1311 — Model Capability Gaps vs Training Needs
model card
A model card is like a nutrition label for AI models.
Lesson 41 — Understanding Model Cards
Model ID
The exact model version (e.g., gpt-4-0613 vs gpt-4-turbo-preview)
Lesson 1400 — Tracking Feedback Metadata
Model naming
Models like claude-3-opus, claude-3-sonnet, and claude-3-haiku are organized by capability tier (not incremental versions)
Lesson 86 — Anthropic Claude API: Constitutional AI Approach
Model outputs
The raw response, token count, and latency
Lesson 873 — Tracking and Logging A/B Test Data
model registry
A model registry is a centralized repository that stores model artifacts alongside critical metadata: training data versions, hyperparameters, performance metrics, deployment status, and lineage (which code/data produced this model).
Lesson 906 — Model Registry IntegrationLesson 1605 — Model Registry PatternsLesson 1610 — Model Registry and Version Management
Model selection impact
Model selection impact is huge: GPT-4 might cost 10-30× more than GPT-3.5 per request.
Lesson 33 — Measuring Cost per Request
Model serving
Model serving is the opposite challenge: taking that trained model and making it available for real-time predictions at scale.
Lesson 1005 — What is Model Serving?
Monitor dependencies
Track which features are provider-specific versus industry-standard (like OpenAI-compatible APIs).
Lesson 1124 — Vendor Lock-in and Migration Strategies
Multi-adapter benchmarking
Multi-adapter benchmarking means running controlled experiments on held-out validation or test data across all candidate adapters:
Lesson 1382 — Multi-Adapter Benchmarking and Selection
multi-cloud strategy
A multi-cloud strategy means running AI workloads across multiple providers (AWS, Azure, GCP, specialized AI platforms) to optimize costs and avoid dependency on a single vendor.
Lesson 1218 — Multi-Cloud and Hybrid Strategies
Multi-tenant key isolation
Multi-tenant key isolation means provisioning separate API credentials for each tenant (or environment, or customer tier).
Lesson 1480 — Multi-Tenant Key Isolation
Multi-user memory isolation
Multi-user memory isolation means architecting your memory systems so each user or session has its own protected memory store.
Lesson 606 — Multi-User Memory Isolation

N

n8n
n8n is the open-source alternative you can self-host.
Lesson 1833 — No-Code Platforms Overview
name
The function's identifier (like get_weather or search_database). Use snake_case and keep it descriptive but concise.
Lesson 555 — Function Schema Structure and OpenAI FormatLesson 761 — Defining Function Schemas
NDCG
NDCG is sophisticated: it considers how relevant each result is (not just yes/no) and where it appears (position matters).
Lesson 797 — Retrieval Quality Metrics
Near-zero waste
Blocks are only allocated as needed, and unused blocks are immediately available
Lesson 1035 — PagedAttention and vLLM
Negative pairs
Negative pairs are items that should have different embeddings:
Lesson 240 — Contrastive Learning for Embeddings
Nested objects
Nested objects are objects inside objects ({ "user": { "address": { "city": "NYC" } } })
Lesson 762 — Nested Objects and Arrays
No API costs
After downloading the model, generating embeddings is free
Lesson 217 — Sentence Transformers Library
No server session storage
The server doesn't maintain session objects or in-memory state between calls
Lesson 921 — Understanding Stateless Architecture in LLM Applications
No user-specific data
The integration doesn't need to act on behalf of individual users
Lesson 1845 — API Key vs OAuth: When to Use Each
Node
A Node is a chunked, indexed piece of a Document.
Lesson 514 — Documents and Nodes: LlamaIndex Data Model
Node affinity
Node affinity is Kubernetes' way of matching pods to nodes based on labels.
Lesson 1109 — Node Affinity and GPU Node Pools
Non-commercial
Non-commercial means personal projects, academic research, or educational purposes only.
Lesson 42 — Model Licensing and Usage Rights
Non-deterministic behavior
The same prompt can produce different outputs. A "successful" API call (200 status) might still return a hallucinated, incorrect, or unusable response. Traditional success/failure metrics miss this entirely.
Lesson 1219 — Why Observability Matters for LLM Systems
Numeric scores
Numeric scores are continuous values, often 0-100.
Lesson 812 — Binary vs Scalar Judgments
NVLink
NVLink is NVIDIA's high-speed interconnect technology, providing 300-600 GB/s bandwidth between GPUs (10-20× faster than PCIe).
Lesson 1079 — Communication Overhead and Bandwidth

O

OAuth
OAuth is a delegation protocol that lets users grant your app limited access to their resources without sharing credentials.
Lesson 1845 — API Key vs OAuth: When to Use Each
Observable state changes
A specific condition is now true (file exists, query answered, approval received)
Lesson 623 — Stopping Conditions: Goal Achievement
Offline capability
Works without internet once models are cached
Lesson 217 — Sentence Transformers Library
ONNX Runtime
ONNX Runtime is a cross-platform inference engine that runs ONNX (Open Neural Network Exchange) models with high performance, regardless of which framework originally trained the model.
Lesson 1652 — ONNX Runtime for Cross-Framework DeploymentLesson 1673 — ONNX Runtime for Cross-Platform Deployment
Out-of-Range Values
A max_tokens value of -50 or a temperature of 5.0 (when valid range is 0.0-2.0).
Lesson 976 — Handling Missing and Invalid Parameters
Overlap
Overlap means duplicating some content between adjacent chunks.
Lesson 341 — Overlap Strategies

P

parallel function calling
This is called parallel function calling.
Lesson 551 — Parallel Function Calls
parameter sweep
A parameter sweep means running the same prompt multiple times while varying generation parameters like temperature, top_p, or max_tokens.
Lesson 203 — Temperature and Parameter Sweeps
Parsing
Parsing means extracting citation markers using pattern matching:
Lesson 365 — Parsing and Validating Citations
Payload
The actual content—task parameters, data, observations, or responses.
Lesson 682 — Message Protocols and Schemas
Performance constraints
Framework overhead is unacceptable for your latency or resource budget
Lesson 712 — Framework Selection and Custom Solutions
Perplexity
Measures how "surprised" the model is by the validation data. Lower is better. Think of it as the model's confusion level.
Lesson 1333 — Evaluation Metrics for Fine-Tuned Models
Persistence
Persistence means saving your fully-built index (with embeddings, nodes, and structure) to disk or external storage, then loading it back instantly when needed.
Lesson 524 — Storage Context and Persistence
PII detection
PII detection is the practice of scanning text for structured patterns that identify real individuals, so you can redact, block, or flag outputs before they reach users.
Lesson 1455 — PII Detection Fundamentals
Pipeline versioning
Pipeline versioning means tracking these changes systematically—using Git for code, tagging DAG versions, and maintaining separate environments for development and production.
Lesson 497 — Pipeline Versioning and Testing
Plan repair
Plan repair is more surgical—modifying specific steps in the existing plan while preserving what's still valid.
Lesson 614 — Replanning and Plan Repair
Plan verification and validation
Plan verification and validation means checking the plan's quality before committing to execution.
Lesson 617 — Plan Verification and Validation
Pod
A Pod is the smallest deployable unit in Kubernetes—typically one or more containers running together.
Lesson 1102 — Kubernetes Core Concepts: Pods, Deployments, Services
Pods
Pods are individual cooking stations, Deployments are the head chef's recipe and staffing plan, and Services are the waiters connecting customers to the kitchen.
Lesson 296 — Pinecone Architecture and ConceptsLesson 1102 — Kubernetes Core Concepts: Pods, Deployments, Services
Positive pairs
Positive pairs are items that should have similar embeddings:
Lesson 240 — Contrastive Learning for Embeddings
Post-filtering
Post-filtering means running the full vector search first, then filtering out results that don't match your metadata requirements afterward.
Lesson 277 — Pre-filtering vs Post-filtering
PostgreSQL with pgvector
PostgreSQL with pgvector is an extension that adds vector operations to the world's most popular open-source relational database.
Lesson 290 — Traditional Databases with Vector Support
Pre-filtering
Pre-filtering is faster when metadata drastically reduces your dataset (searching 1,000 items instead of 1,000,000).
Lesson 277 — Pre-filtering vs Post-filtering
Pre-load at startup
Load quantized weights during container initialization, not on first request—cold starts are more expensive with quantized models
Lesson 1048 — Production Deployment of Quantized Models
Predictive Parity
Positive predictions are equally accurate across groups. If your model says "approve," it should be correct 90% of the time regardless of group membership.
Lesson 1565 — Defining Fairness in AI Systems
presence penalty
Frequency penalty and presence penalty are numerical parameters (typically ranging from -2.0 to 2.0) that modify how likely the model is to reuse tokens:
Lesson 142 — Frequency and Presence Penalties
Preserve base capabilities
The base model's general knowledge remains intact
Lesson 1384 — Domain Adaptation with PEFT
Pricing iteration
Pricing iteration means analyzing production metrics like API calls per user, token consumption patterns, feature adoption rates, and cost per interaction to adjust your tiers, limits, and packaging.
Lesson 1886 — Pricing Iteration Based on Usage Patterns
primary metric
Your primary metric is the single success criterion that determines whether variant B wins.
Lesson 1862 — Metrics Selection for AI A/B Tests
Primary metrics
Primary metrics are your north star—the single most important measure of success.
Lesson 870 — Choosing Metrics for AI A/B Tests
privacy budget
A privacy budget is the total amount of epsilon you're willing to "spend" across all queries or operations.
Lesson 1536 — Epsilon and Privacy Budgets
production rule
Think of a CFG as a recipe book where each recipe (called a production rule) tells you how to build valid pieces of text.
Lesson 778 — Context-Free Grammars (CFG) Basics
Progress tracking
Progress tracking means showing real-time updates about how much work is done (like a progress bar: "Processing document 1,247 of 10,000").
Lesson 485 — Progress Tracking and Checkpointing
Progressive Generation
Break input text into natural boundaries (sentence endings, punctuation) and synthesize each segment independently. Start playback as soon as the first chunk is ready.
Lesson 1709 — Real-Time TTS and Audio Synthesis
Prometheus
Prometheus is a monitoring system that scrapes metrics from your application endpoints.
Lesson 1126 — Custom Metrics and Prometheus for AI Scaling
Prompt details
The exact prompt template and variables used
Lesson 873 — Tracking and Logging A/B Test Data
prompt injection detection classifier
A prompt injection detection classifier is a specialized model (often a smaller, faster ML model) trained specifically to recognize patterns that indicate injection attempts.
Lesson 1447 — Prompt Injection Detection Classifiers
Prompt stuffing
Prompt stuffing means directly including all your context in the prompt itself:
Lesson 328 — RAG vs Prompt Stuffing
Prompt versioning
Prompt versioning means treating each prompt like software code: assign it a version number, track every change, and maintain a history so you can always return to a previous version if needed.
Lesson 202 — Prompt Versioning and Change Management
Prompts and completions
The exact input text and generated outputs for every request
Lesson 1267 — Weights & Biases for LLM Tracking
Prosody
Prosody refers to the rhythm, stress, and intonation of speech.
Lesson 1719 — Emotion and Prosody Analysis
prototyping architecture
Similarly, prototyping architecture means starting with managed APIs (like OpenAI, Anthropic, or Cohere) that let you build and iterate in days, not months.
Lesson 29 — Prototyping vs Production Architecture
Pydantic
Pydantic is a Python library that solves this through data validation using Python type hints.
Lesson 765 — Pydantic Basics for LLM Output
PyPDF2
PyPDF2 is lightweight and fast, ideal for simple text extraction and reading metadata (author, creation date, page count).
Lesson 457 — PDF Extraction Fundamentals
PySyft
PySyft is the powerhouse for federated learning, enabling you to simulate multi-party computation, secure aggregation, and encrypted training across distributed datasets without centralizing data.
Lesson 1544 — Practical Tools and Frameworks

Q

Quality Problems
Responses that are off-topic, too verbose, poorly formatted, or miss key information from the prompt. Compare what you asked for versus what you got.
Lesson 1296 — Analyzing Prompt-Response Pairs
Quantitative constraints
Quantitative constraints are your primary levers:
Lesson 1881 — Free Tier and Freemium Strategy
Query classification and routing
Query classification and routing means analyzing the user's question before retrieval, categorizing it by type, and then directing it to the most appropriate retrieval strategy.
Lesson 375 — Query Classification and Routing
Query rewriting based on previous results
Query rewriting based on previous results means looking at what the LLM produced, identifying what's missing or unclear, and reformulating your query to retrieve additional context that addresses those gaps.
Lesson 440 — Query Rewriting Based on Previous Results
Queue depth
If requests are backing up in your queue, you need more replicas
Lesson 1125 — Horizontal Pod Autoscaling for AI Workloads

R

RAG
RAG means indexing those documents, retrieving only the relevant ones, then including just those in your prompt.
Lesson 328 — RAG vs Prompt Stuffing
RBAC for agents
RBAC for agents means defining explicit permissions that map each agent's role to:
Lesson 677 — Role-Based Access Control for Agents
Read-only by default
Functions should only retrieve data unless write access is absolutely necessary
Lesson 1450 — Sandboxing and Least Privilege for Tools
Readiness probe
Checks if your model is loaded and can handle requests (e.g., model warm-up complete)
Lesson 1618 — Health Checks and Graceful Shutdown
reading traces
Once your observability tools are collecting data, the real work begins: reading traces means analyzing the captured execution flow to understand what's happening inside your LLM system.
Lesson 1293 — Reading LLM Traces in Production
Reasoning about recency
The LLM can favor newer information
Lesson 358 — Metadata Injection Patterns
Reference Counting
Track how many active requests are using each adapter to avoid evicting one that's currently in use.
Lesson 1376 — Adapter Caching and Warm-Up
Refusal behavior
Refusal behavior is how your model says "no" to harmful requests—but the challenge is ensuring it doesn't refuse too much (becoming unusable) or too little (becoming unsafe).
Lesson 1468 — Evaluating Refusal Behavior
regression suite
A regression suite is a version-controlled collection of test cases that you run repeatedly to ensure your LLM system doesn't get worse over time.
Lesson 829 — What is a Regression Suite for LLM Systems
Regression testing
Regression testing means re-running a suite of test cases after every change to ensure old capabilities still work.
Lesson 668 — Regression Testing and Agent Versioning
Relationships to nearby words
The embedding changes based on what's around it
Lesson 210 — Contextual vs Static Embeddings
relevance
In RAG systems, relevance means how well a retrieved chunk actually answers or supports the user's query.
Lesson 423 — Understanding Relevance in RAG Context
Relevance scoring
Track how often each memory is retrieved or referenced. Memories never accessed after initial storage score low and become pruning candidates. This works like cache eviction policies in computer systems.
Lesson 604 — Forgetting and Memory Pruning
Replanning
Replanning means generating an entirely new plan when the current one becomes unworkable.
Lesson 614 — Replanning and Plan Repair
Reproducibility
The exact same model weights and behavior everywhere
Lesson 1597 — Understanding Model Serialization
Request Isolation
Even when batching requests across adapters (as we learned previously), ensure logs, metrics, and error traces are partitioned by tenant. A bug in Customer A's adapter shouldn't expose Customer B's prompts or outputs.
Lesson 1375 — Multi-Tenant Adapter Serving
Requests
Requests are what Kubernetes uses to decide which node can host your pod—it's like reserving a hotel room.
Lesson 1105 — Resource Requests and Limits for GPU Workloads
Requirements changed
The behavior being tested is no longer desired
Lesson 838 — Maintaining and Evolving Your Regression Suite
Reranking
Reranking is your second pass—a more computationally expensive but precise model that rescores the top candidates.
Lesson 1762 — Multimodal Reranking Strategies
Resource Efficiency
AI models are resource-hungry. Kubernetes intelligently schedules containers onto nodes with available GPU resources, maximizes hardware utilization, and prevents resource conflicts between multiple models.
Lesson 1101 — What is Kubernetes and Why for AI?
RestrictedPython
RestrictedPython is Python's answer to safe code execution.
Lesson 1499 — Language-Specific Sandbox Tools
Retrieval Quality
Given a specific query, assert that the correct documents (or chunks) are retrieved. Check both relevance and ranking.
Lesson 893 — Testing Complete RAG Pipelines
Retry Strategies
Some failures are transient (network hiccups, temporary file locks). Implement exponential backoff for retries—wait longer between each attempt, with a maximum retry limit.
Lesson 476 — Error Handling and Logging in Parsers
Reusable patterns
Common patterns like RAG, prompt chaining, and agent loops are pre-built.
Lesson 499 — What is LangChain and Why Use It
reward hacking
Models can become extremely creative at gaming the reward function in ways you never intended—a phenomenon called reward hacking.
Lesson 1417 — RLHF Safety and Alignment
RGB vs BGR
OpenCV loads images in BGR by default, but most deep learning frameworks expect RGB. The channels are simply reversed: [Blue, Green, Red] becomes [Red, Green, Blue].
Lesson 1641 — Color Space Conversions
Rollback strategies
Before deploying, define how you'll undo it. Keep the previous model version warm in a blue-green setup, or use canary deployments with automatic rollback triggers. Document the rollback procedure—when stakes are high, you won't remember the commands.
Lesson 918 — Rollback Strategies and Circuit BreakersLesson 1016 — Production Deployment Checklist
rubric
A rubric is a structured scoring guide that breaks down "quality" into specific, measurable dimensions.
Lesson 840 — Designing Evaluation Rubrics

S

Sampling
Sampling is the solution: systematically capturing only a representative subset of your telemetry data while maintaining statistical confidence in your insights.
Lesson 1228 — Sampling Strategies for High-Volume SystemsLesson 1288 — Sampling Strategies for High-Volume Systems
sandbox
A sandbox is an isolated execution environment where code runs with restricted access to system resources.
Lesson 1495 — Why Sandboxing for Code Generation
Sandboxing
Sandboxing means creating an isolated, restricted environment where code runs with limited permissions.
Lesson 652 — Sandboxing Python Code Execution
Sanitizing
Sanitizing means removing or replacing dangerous content entirely.
Lesson 154 — Escaping and Sanitizing User Input
Scalability matters
Adding new capabilities means adding new agents, not rebuilding one massive system
Lesson 669 — Introduction to Multi-Agent Systems
schema
A schema is the formal specification that validates messages, ensuring they contain the right information in the right shape.
Lesson 682 — Message Protocols and Schemas
Schema validation
Rules that check whether a message is well-formed before processing. Think of it like a bouncer checking IDs—malformed messages get rejected early, preventing downstream errors.
Lesson 682 — Message Protocols and Schemas
Schema versioning
Schema versioning means explicitly tracking different versions of your data structure, like software releases.
Lesson 790 — Schema Evolution and Versioning
Secret management services
Secret management services are purpose-built systems that centralize, encrypt, rotate, and audit access to sensitive credentials.
Lesson 1475 — Secret Management Services
seed
A seed is a number that initializes the random number generator used during text generation.
Lesson 143 — Seed for Reproducible Generation
Segmenting feedback by user cohorts
Segmenting feedback by user cohorts means grouping feedback based on shared user characteristics—experience level, use case, subscription tier, industry, or engagement patterns—then analyzing each group separately to find segment-specific issues and opportunities.
Lesson 865 — Segmenting Feedback by User Cohorts
Seldon Core
Seldon Core is Kubernetes-native and framework-agnostic.
Lesson 1607 — Serving Frameworks Overview
Selective retention
Selective retention means keeping critical messages (like system prompts, key user preferences, or important facts) while removing less relevant turns.
Lesson 740 — Selective Message Retention Strategies
sentinel value
A sentinel value is a special marker in your stream that signals an error has occurred.
Lesson 999 — Error Handling in Streaming Responses
Separation of duties
Separation of duties means the people operating the AI system shouldn't be the same ones auditing it.
Lesson 1513 — Access Control for Audit Logs
Sequential
Sequential means you wait for each person's drink before ordering the next one.
Lesson 1162 — Async/Await and Concurrent API Calls
Serialization
Serialization is the process of converting your validated Pydantic model back into JSON—the format LLMs understand.
Lesson 774 — Model Configuration and Serialization
server-side
The server-side (your backend) is the restricted back office where sensitive operations happen.
Lesson 1478 — Client-Side vs Server-Side Keys
Services
Pods are individual cooking stations, Deployments are the head chef's recipe and staffing plan, and Services are the waiters connecting customers to the kitchen.
Lesson 1102 — Kubernetes Core Concepts: Pods, Deployments, Services
Session identity
Session identity is the unique identifier (like a session ID) that labels one user's conversation thread.
Lesson 715 — Session Identity and User Tracking
Session storage
Session storage means persisting conversation data beyond the lifetime of a single request.
Lesson 741 — Session Management and Persistence
shadow model
A shadow model is a candidate model that runs in parallel with your production model, receiving the same inputs but with its outputs silently logged instead of served to users.
Lesson 1614 — A/B Testing with Model Shadows
Shadow Testing
Shadow Testing means running the new model version in parallel with your production model.
Lesson 836 — Shadow Testing and Canary Deployments
Sharding
Sharding means splitting your data across multiple separate databases, while partitioning divides data within a single database into smaller, manageable chunks.
Lesson 950 — Database Sharding and Partitioning Strategies
signatures
Instead of writing prompts, you write modular programs using composable building blocks called signatures and modules.
Lesson 529 — DSPy: Programming LLM Pipelines
Similarity matching
Beyond exact matches, consider semantic similarity caching—if two prompts are 95% similar, maybe they deserve the same cached response.
Lesson 1156 — Prompt-Level Caching Strategies
similarity score threshold
A similarity score threshold is a minimum value you require before returning a result to the user.
Lesson 235 — Similarity Score Thresholds
Simpler requirements
The third-party doesn't support OAuth or you need quick prototyping
Lesson 1845 — API Key vs OAuth: When to Use Each
Simplified features
Design features that are fast to compute in real-time from the start
Lesson 1619 — Feature Engineering vs. Feature Serving
Single-example validation
Single-example validation is like tasting one spoonful of soup and declaring the entire pot perfect.
Lesson 197 — Why Test Prompts: Beyond Intuition
Skip-frame strategies
Sometimes processing every 3rd frame is acceptable
Lesson 1661 — Video Inference vs Single-Image Inference
Slash Commands
Slash Commands are user-invoked shortcuts like /summarize or /ask-ai.
Lesson 1821 — Slack Event Handling and Commands
Small datasets or infrequent queries
Sorting is fine and simpler to implement
Lesson 231 — Top-K Retrieval Implementation
Small sample challenge
Intersectional groups are often underrepresented in datasets, making both training and evaluation harder
Lesson 1563 — Intersectionality and Compounding Bias
Smaller dimensions
Smaller dimensions are faster and cheaper but may miss subtle distinctions.
Lesson 219 — Model Selection Criteria
Source Panels
A dedicated sidebar or bottom section listing all cited sources with thumbnails, titles, and links. Keeps the main text clean while centralizing references. Users can browse sources independently.
Lesson 366 — Citation Display Patterns
span
A span is an individual unit of work within a trace.
Lesson 1223 — Distributed Tracing Fundamentals
Special tokens
Special tokens are markers the model needs to understand structure.
Lesson 52 — Tokenizers: Encoding and Decoding
Spot instances
Spot instances are unused cloud capacity offered at 60-90% discounts.
Lesson 1069 — Cloud GPU Options and Spot Instances
SQL Generation
An LLM creates database queries based on natural language requests. A malicious user might prompt: "Show me all users where name is 'admin'--' OR '1'='1". If the LLM naively converts this to SQL, it could expose the entire user table.
Lesson 1492 — SQL and Code Injection in LLM Contexts
state dict
A state dict is a Python dictionary that maps each layer to its parameter tensor.
Lesson 1602 — PyTorch State Dicts and Checkpoints
state machine
A state machine is a computational pattern that models a system as a collection of distinct states with well-defined transitions between them.
Lesson 1777 — What Are State Machines and Why Use Them in AI?
State management
Variables, intermediate results, and context are preserved across failures without manual database writes
Lesson 489 — Pipeline Orchestration FundamentalsLesson 499 — What is LangChain and Why Use ItLesson 1798 — Temporal for AI Workflows
State persistence
State persistence means serializing (saving) your state machine's current state, context data, and transition history to durable storage so you can reload and continue exactly where you left off.
Lesson 1785 — State Persistence and Resumption
State pruning
State pruning is the practice of selectively removing or compressing parts of your agent's accumulated state while preserving what matters most for decision-making.
Lesson 625 — State Pruning and Memory Management
Stateless execution
Stateless execution means no side effects persist between runs.
Lesson 1497 — Serverless Functions as Sandboxes
Stateless LLM Layer
Each API call to your LLM is independent. No conversation history lives in the model service itself. This keeps your backend simple, scalable, and easy to load balance.
Lesson 928 — Hybrid Architectures: Best of Both Worlds
Static Asset Caching
Tokenizer files, configuration JSONs, and other static artifacts get cached at CDN edge nodes. These small files are retrieved in milliseconds rather than seconds.
Lesson 1132 — Regional Model Caching and CDN Strategies
Static thresholds
Static thresholds are fixed values you set based on requirements or experience:
Lesson 1254 — Threshold-Based Alerting
Statistical power
Statistical power is your ability to detect a real performance difference when it exists.
Lesson 827 — Dataset Size and Statistical Power
stop sequence
A stop sequence is a specific string of text that tells the model "stop generating immediately when you encounter this." Think of it like setting a finish line in a race—the moment the model writes that exact sequence, generation halts, and everything up to (but not including) that marker is returned.
Lesson 141 — Stop Sequences and Early Termination
Stratified sampling
Stratified sampling means dividing your data into meaningful groups (strata) and sampling from each group proportionally—or deliberately over-sampling rare but important cases.
Lesson 823 — Sampling Strategies for Coverage
Stream metadata headers
Stream metadata headers are HTTP headers sent at the beginning of a streaming response that carry important context about the request and the AI system serving it.
Lesson 1004 — Stream Metadata and Version Headers
Stripping HTML
Stripping HTML means removing tags while keeping text content.
Lesson 469 — HTML and Markdown Cleaning
Structured data extraction
Structured data extraction means capturing not just content, but its organization: table cells with their headers, document sections with their hierarchy, and metadata like authors or creation dates — all while preserving how these elements relate to each other.
Lesson 468 — Structured Data Extraction from Documents
Structured output
Structured output means forcing the LLM to return data in a predictable format (JSON, XML, etc.) instead of casual prose.
Lesson 755 — Why Structured Output Matters
Structuring
Organizing scattered data into clear fields
Lesson 587 — Observation Space and Input Processing
Subjective but pattern-based criteria
Tasks like tone assessment, coherence checking, or instruction following where patterns are recognizable
Lesson 808 — When to Use LLM-as-a-Judge
Subtle style rules
Subtle style rules are hard to capture in prompts (sentence structure preferences, vocabulary choices)
Lesson 1308 — Style, Tone, and Format Consistency
Sudden spikes
A user making 100x their normal requests, possibly indicating a runaway loop or intentional abuse
Lesson 1247 — Anomaly Detection in Token Usage Patterns
Synchronous blocking
The client waits for the response—no queueing
Lesson 1634 — Online Serving with REST APIs
Synchronous execution
Synchronous execution means calling tools one at a time, waiting for each to complete before starting the next.
Lesson 592 — Synchronous vs Asynchronous Execution
System partial
The AI's role and general behavior rules
Lesson 153 — Prompt Partials and Composition

T

Tagging for lifecycle tracking
Tagging for lifecycle tracking is your first defense.
Lesson 1217 — Idle Resource Detection and Cleanup
Task
A specific piece of work that needs completion. Tasks have descriptions, expected outputs, and can be assigned to specific agents. They're like work items in a project management system.
Lesson 704 — CrewAI Framework Fundamentals
Task decomposition
Task decomposition is the process of splitting a complex goal into sequential subtasks the agent can actually accomplish with its available tools.
Lesson 127 — Task Decomposition and Step-by-Step InstructionsLesson 609 — Task Decomposition FundamentalsLesson 694 — Task Decomposition and Distribution
Tasks
Tasks are the individual operations within a flow—chunking documents, calling an embedding API, inserting vectors.
Lesson 491 — Prefect for Modern AI Workflows
TensorFlow Lite
TensorFlow Lite is the streamlined version designed specifically for these constrained environments, trading some flexibility for dramatically reduced size and faster inference.
Lesson 1676 — TensorFlow Lite for Mobile and Embedded
TensorFlow Serving
TensorFlow Serving is Google's battle-tested framework designed specifically for TensorFlow models (SavedModel format).
Lesson 1015 — Framework ComparisonLesson 1607 — Serving Frameworks OverviewLesson 1651 — TensorFlow Serving for Vision
TensorRT
After converting your model to ONNX format (from the previous lesson), TensorRT is NVIDIA's high-performance deep learning inference optimizer specifically designed for their GPU hardware.
Lesson 1674 — TensorRT for NVIDIA Hardware
Test Flakiness Detection
Flag tests that intermittently fail. For AI systems, distinguish between acceptable model variance and genuine flakiness. A RAG test that occasionally retrieves slightly different documents might be expected; one that randomly times out is not.
Lesson 910 — CI Monitoring and Debugging Failures
Testing with mock data
Testing with mock data means creating fake but realistic sample variables, rendering your template with them, and checking that the final prompt looks right.
Lesson 156 — Testing Templates with Mock Data
text-embedding-ada-002
The previous standard model producing 1,536-dimensional vectors
Lesson 215 — OpenAI Embeddings API Overview
Think of it like
An automated content reviewer that flags images just as a human moderator would, but instantly and at scale.
Lesson 1743 — Safety and Content Filtering for Images
Think of it like plumbing
Each component (prompt template, LLM, output parser) is a pipe segment. The | operator connects them, and data flows through automatically from left to right.
Lesson 507 — LCEL: LangChain Expression Language
Think of it like this
Training data is like ingredients in a recipe. You need to check if each ingredient is safe, legally sourced, and allowed for your intended dish—you can't "un-bake" problematic ingredients later.
Lesson 1324 — Data Privacy and Licensing
Third-party audits
Third-party audits are structured engagements where you hire specialized security firms to systematically probe your LLM application for vulnerabilities—prompt injections, content filter bypasses, PII leakage, jailbreaks, and more.
Lesson 1472 — Third-Party Security Audits and Bug Bounties
threshold
A threshold is the cutoff point where you decide to take action.
Lesson 1433 — Confidence Scores and Thresholding
Throughput
Throughput is the number of retrieval requests your system can process per unit of time—typically queries per second (QPS).
Lesson 411 — Latency and Throughput Metrics
Tie-breaking
Allow the judge to declare ties when outputs are equally good
Lesson 813 — Comparative Evaluation (Pairwise)
Tiered storage
Tiered storage means matching data access patterns to storage types.
Lesson 952 — Storage Cost Optimization and Data Lifecycle
Time-of-day irregularities
Heavy usage at 3 AM when your users are typically asleep
Lesson 1247 — Anomaly Detection in Token Usage Patterns
TLS Handshake
The secure encryption negotiation that follows. This adds another 50-100ms for certificate exchange and cipher agreement. Modern APIs use connection pooling to avoid repeating this for every request.
Lesson 1140 — Network Latency and API Response Times
Token budget
Your context window is finite; examples crowd out actual content
Lesson 1307 — Latency and Token Budget Constraints
Token budget allocation
Token budget allocation means consciously deciding how many tokens to dedicate to:
Lesson 1153 — Token Budget Allocation
Tone and Style Guidance
Tone and Style Guidance means explicitly telling the model how to write, not just what to write.
Lesson 134 — Tone and Style Guidance
Tool functions
The actual callable functions you've defined
Lesson 589 — Action Space and Tool Calling
TorchServe
TorchServe is the PyTorch-native solution.
Lesson 1607 — Serving Frameworks Overview
trace
A trace represents the complete end-to-end journey of a single request through your system.
Lesson 1223 — Distributed Tracing Fundamentals
Tracing
Tracing means capturing a detailed record of every tool call—its name, input arguments, output, and the surrounding context at that moment—so you can replay and debug exactly what happened.
Lesson 660 — Tracing Tool Calls and ContextLesson 1138 — Tracing Multi-Step LLM Chains
Training needs
Training needs are situations where the model could perform the task but needs examples to learn your specific requirements—like adopting your company's writing style, following domain-specific formatting rules, or using specialized terminology correctly.
Lesson 1311 — Model Capability Gaps vs Training Needs
Transparency
Rules are readable and auditable
Lesson 1595 — Prompt-Based Alignment Strategies
Type Constraints
The field type itself (str, int, bool) is your first filter.
Lesson 766 — Defining Field Types and Constraints
Type safety
Numbers are numbers, strings are strings—no guessing
Lesson 760 — Function Calling for Structured Output

U

Uniform Sampling
Uniform Sampling is the simplest strategy: extract frames at regular intervals (e.g., every 5th frame or 1 frame per second).
Lesson 1662 — Frame Extraction and Sampling Strategies
Uniqueness percentage
Fraction of records that are singletons
Lesson 1533 — Re-identification Risk Assessment
Updating Records
PATCH or PUT requests with the record ID and changed fields. If your AI detects a customer is at risk, it might update an opportunity.stage or add a note to contact.description.
Lesson 1809 — Reading and Writing CRM Data
Usage Alerts
Usage Alerts are notifications triggered when your token consumption or costs exceed predefined thresholds.
Lesson 1182 — Setting Usage Alerts and Budgets
Usage Growth
Visualize active users, request volumes, and adoption rates over time. Highlight which features drive engagement and where growth is accelerating or stalling.
Lesson 1259 — Executive and Business Dashboards
User
Think of it like a play: the system message is the stage directions, the user is one character speaking, and the assistant is another character responding.
Lesson 91 — System, User, and Assistant Message Roles
User input
User input is treated as potentially adversarial content to be processed, not executed
Lesson 1445 — Instruction Hierarchy and Privilege Separation
User instruction partial
The actual question or request
Lesson 153 — Prompt Partials and Composition
User messages
User messages are the actual queries or prompts you want answered
Lesson 91 — System, User, and Assistant Message Roles
UTF-8
UTF-8 is the universal translator—it can represent nearly every character from every language.
Lesson 470 — Character Encoding and Unicode Handling
Utility loss
The percentage-point drop in F1, accuracy, or whatever metric matters
Lesson 1539 — Trade-offs: Privacy vs Accuracy

V

Validation checks
The result passes predefined quality or completeness tests
Lesson 623 — Stopping Conditions: Goal Achievement
values
The vector itself (if requested)
Lesson 299 — Querying and Filtering in Pinecone
variance
This variance means you need larger samples to confidently say "Model B is actually better than Model A."
Lesson 871 — Statistical Power and Sample Size for AI Tests
Vector
The numerical embedding (must match your index's dimension)
Lesson 298 — Upserting Vectors to Pinecone
Vector data
The embeddings themselves
Lesson 320 — Backup and Disaster Recovery
Version control
Track which model version is deployed
Lesson 1597 — Understanding Model Serialization
Versioned test cases
A collection of tasks your agent should complete (e.g., "book a flight," "calculate compound interest")
Lesson 668 — Regression Testing and Agent Versioning
Visibility
See which tasks succeeded, failed, or are running
Lesson 490 — Apache Airflow for AI Pipelines

W

W&B Tables
W&B Tables are interactive, spreadsheet-like visualizations that let you organize and compare LLM experiments in a structured format.
Lesson 1268 — W&B Tables for Prompt Comparison
Warm-up
Preload adapters you know will be popular before traffic arrives. This is especially useful after deployment—load your top-5 customer adapters during initialization so the first requests don't experience cold-start delays.
Lesson 1376 — Adapter Caching and Warm-Up
Wasted resources
If sequences are shorter than the max length, unused memory sits idle
Lesson 1032 — Static vs Dynamic KV Cache Allocation
Weaviate
Weaviate is the Swiss Army knife—it's not just a vector database but a full semantic search engine with built-in vectorization modules.
Lesson 289 — Open Source Vector Databases
Webhook handlers
Webhook handlers are HTTP endpoints that receive and validate platform events.
Lesson 1819 — Communication Platform Bot Fundamentals
Webhook Reliability
Communication platforms send HTTP POST requests to your bot's endpoint. If your server is down or responds slowly, the platform may retry (causing duplicate processing) or drop the event entirely. Always:
Lesson 1827 — Bot Deployment and High Availability
Why it failed
the specific validation rule or type mismatch
Lesson 578 — Error Messages for LLMs
Window memory
For reactive tasks where only recent turns are relevant (quick Q&A, transactional bots)
Lesson 510 — Memory: Summary and Window Memory
word-level alignment
This is called word-level alignment or forced alignment, and it's essential for:
Lesson 1688 — Timestamp and Word-Level Alignment

X

XState
XState is the most popular state machine library in the JavaScript/TypeScript ecosystem.
Lesson 1780 — State Machine Libraries: XState and Python Alternatives

Z

Zapier
Zapier is the most user-friendly option with thousands of pre-built app integrations.
Lesson 1833 — No-Code Platforms Overview