AI Engineering Glossary
Key terms from the AI Engineering course, linked to the lesson that introduces each one.
507 terms.
A
- Abstraction layers
- Abstraction layers are your primary defense.
- Lesson 1124 — Vendor Lock-in and Migration Strategies
- Accelerate
- Accelerate is Hugging Face's library that abstracts away the complexity of distributed computing.
- Lesson 1076 — Setting Up Multi-GPU with Accelerate
- action space
- An action space is the catalog of all possible actions your AI agent can perform in response to its observations and reasoning.
- Lesson 589 — Action Space and Tool Calling
- Active Requests
- The number of in-flight LLM calls at this moment. A sudden spike might indicate a retry storm or API issues.
- Lesson 1258 — Real-Time Monitoring Dashboards
- Adaptation
- Adaptation means modifying them strategically:
- Lesson 825 — Public Benchmarks and Adaptation
- Adapter caching
- Adapter caching means keeping recently-used or frequently-accessed adapters in GPU or CPU memory so they're immediately available when the next request arrives.
- Lesson 1376 — Adapter Caching and Warm-Up
- adapter registry
- An adapter registry is like a library catalog system for your adapters.
- Lesson 1370 — Adapter Registry and Management
- adaptive cards
- Unlike simpler webhook-based bots (Slack, Discord), Teams bots leverage a stateful conversation framework that handles complex multi-turn dialogues, persistent context, and rich interactive components called adaptive cards.
- Lesson 1823 — Microsoft Teams Bot Framework
- Adding noise
- Adding noise means injecting small, random distortions into the results to make it mathematically impossible to infer private details about any single person.
- Lesson 1537 — Adding Noise to Model Outputs
- Agent
- An individual team member with a specific role, goal, and backstory. Think of agents as specialists—a researcher, writer, or analyst—each with their own capabilities and personality defined in their system prompt.
- Lesson 704 — CrewAI Framework Fundamentals
- Agent memory
- Agent memory is the component that allows an AI agent to store and recall information from previous interactions, observations, and decisions.
- Lesson 595 — What Is Agent Memory?
- agent registry
- In multi-agent systems, an agent registry is that directory.
- Lesson 676 — Agent Registry and Discovery
- Agent state
- The conversation history or context window at that moment
- Lesson 660 — Tracing Tool Calls and Context
- AI agent
- An AI agent is an autonomous system that continuously perceives its environment, makes decisions based on reasoning, and takes actions to achieve specific goals—without needing step-by-step human instructions for every move.
- Lesson 585 — What is an AI Agent?
- AI alignment
- AI alignment is the challenge of ensuring AI systems act according to human values, intentions, and preferences—not just the narrow metrics we measure.
- Lesson 1587 — What is AI Alignment
- AIF360
- Fairlearn (Microsoft) and AIF360 (IBM) are the two most widely adopted fairness toolkits.
- Lesson 1574 — Fairness Metrics Implementation and Tools
- Allowlisting
- Allowlisting means explicitly defining what's allowed and blocking everything else.
- Lesson 1502 — Allowlisting Safe Libraries and APIs
- Alpha
- Alpha is a scaling factor that controls how strongly the adapter's updates influence the base model.
- Lesson 1349 — LoRA Hyperparameters: Rank and Alpha
- Analogy
- Think of consent like a library card. Opt-in means you must sign up to borrow books. Opt-out means everyone gets a card automatically but can return it. Explicit is signing a form; implicit is walking into the library.
- Lesson 611 — ReAct Planning PatternLesson 947 — Vector Database Integration PatternsLesson 1373 — Batching Across AdaptersLesson 1545 — Consent Models for AI Training Data
- Analytics
- Measuring speaking pace, detecting pauses, or highlighting keywords
- Lesson 1688 — Timestamp and Word-Level Alignment
- Annotator training and calibration
- Annotator training and calibration is the systematic process of teaching raters exactly what to look for and keeping their judgments aligned with your standards and each other.
- Lesson 843 — Annotator Training and CalibrationLesson 854 — Annotator Training and Calibration
- Anonymization
- Anonymization is the irreversible removal or transformation of identifying information.
- Lesson 1525 — Anonymization vs Pseudonymization: Key Differences
- Apache Airflow
- Apache Airflow is the veteran—widely adopted for data pipelines and batch AI workflows.
- Lesson 1797 — Orchestration Frameworks Overview
- API key
- An API key is like a special password that identifies your application to an external service.
- Lesson 1473 — API Keys in AI Applications
- API keys
- API keys are simple shared secrets—like a master password to your service.
- Lesson 1845 — API Key vs OAuth: When to Use Each
- Append citations programmatically
- If the answer is factually correct but uncited, inject citations yourself based on chunk relevance scores
- Lesson 367 — Handling Missing or Hallucinated Citations
- Arize
- Arize is built for ML observability and drift detection.
- Lesson 1282 — Comparing Arize and Helicone Use Cases
- array
- A list of items, like ["apple", "banana", "cherry"]
- Lesson 556 — Parameter Types and Required vs Optional Fields
- Assistant
- Think of it like a play: the system message is the stage directions, the user is one character speaking, and the assistant is another character responding.
- Lesson 91 — System, User, and Assistant Message Roles
- Asynchronous execution
- Asynchronous execution means initiating multiple tool calls at once and gathering results as they complete.
- Lesson 592 — Synchronous vs Asynchronous Execution
- Asynchronous processing
- Asynchronous processing means you don't wait for one frame to finish completely before starting the next.
- Lesson 1664 — Real-Time Video Processing Pipelines
- Audience targeting
- Audience targeting means explicitly telling the model who the intended reader is, so it adjusts its language, depth, and style accordingly.
- Lesson 133 — Audience Targeting
- Audit trails
- Log where each piece of data is stored and processed (building on lesson 1523)
- Lesson 1524 — Regional Data Residency and Compliance
- auto
- Normal chatbot interactions where context determines if tools are needed
- Lesson 552 — Forcing and Disabling Function Calls
- AutoClasses
- AutoClasses are smart wrappers that automatically detect and load the correct model architecture for you.
- Lesson 51 — Understanding AutoClasses
- Automatic state management
- The chain handles passing data between steps
- Lesson 506 — Sequential Chains
- Available Tools
- The functions or capabilities the agent can use (from your function registry)
- Lesson 629 — Setting Up the Initial State
- Average Rating
- For explicit thumbs-up/down or star ratings, compute means across time windows (daily, weekly). Track this by feature, user segment, or model version to spot performance differences.
- Lesson 1401 — Aggregating and Analyzing Feedback
B
- Backstories
- Context that shapes the agent's behavior and expertise (e.g., "You are a senior data analyst with 10 years of experience in market research")
- Lesson 705 — Defining Crews and Assigning Roles in CrewAI
- Backward compatibility
- Backward compatibility means newer feature pipelines can still serve older models by maintaining transformations for previous schema versions.
- Lesson 790 — Schema Evolution and VersioningLesson 1002 — Backward Compatibility and DeprecationLesson 1603 — Version Control for Serialized ModelsLesson 1629 — Feature Versioning and Backward Compatibility
- BakLLaVA
- LLaVA (Large Language and Vision Assistant) and BakLLaVA are two leading open-source VLMs you can download and run locally for image understanding tasks like captioning, visual question answering, and multi-turn conversations about images.
- Lesson 1726 — Open-Source VLMs: LLaVA and Bakllava
- base model
- A base model is trained on general data without targeting any specific task.
- Lesson 45 — Model Variants and Checkpoints
- Baseline metrics and regression suites
- Baseline metrics and regression suites are your scientific control group—they let you measure real impact, not just hope for it.
- Lesson 1422 — Evaluation Before and After Model Updates
- Batch processing
- Batch processing means grouping multiple documents together and sending them in a single API call or processing them together locally.
- Lesson 59 — Batch Processing and DataLoadersLesson 220 — Batch Processing for EmbeddingsLesson 477 — Batch Processing Fundamentals
- Batch search
- Batch search means bundling multiple queries into a single request, allowing the system to optimize execution and reduce network overhead.
- Lesson 271 — Batch Search and Query Optimization
- Batch size
- Batch size is the number of inputs you process simultaneously in a single forward pass through the model.
- Lesson 64 — Batch Size and ThroughputLesson 1071 — Batch Size and Throughput Planning
- batch window
- A batch window is the maximum time to wait for requests to accumulate before processing.
- Lesson 1611 — Batching Strategies for Throughput
- BeautifulSoup
- BeautifulSoup is a Python library that parses HTML and lets you navigate the document structure like a tree.
- Lesson 460 — Web Content and HTML Extraction
- Better reasoning
- The LLM can focus purely on strategic thinking without worrying about tool execution
- Lesson 610 — Plan-and-Execute Architecture
- BF16
- FP16 is common on NVIDIA GPUs, while BF16 (bfloat16) is newer and handles extreme values better, making it safer for many models without careful tuning.
- Lesson 70 — Mixed Precision Inference
- Bias in AI systems
- Bias in AI systems refers to systematic errors or unfair outcomes that consistently affect specific groups in model predictions or outputs.
- Lesson 1555 — What is Bias in AI Systems
- Binary judgments
- Binary judgments are yes/no or pass/fail decisions.
- Lesson 812 — Binary vs Scalar Judgments
- blackboard
- In multi-agent systems, a blackboard is a centralized data structure (like a dictionary, database, or object store) where agents post observations, partial results, hypotheses, or task updates.
- Lesson 697 — Blackboard Architecture for Shared State
- blackboard architecture
- A blackboard architecture is a formal pattern where:
- Lesson 681 — Shared Memory and Blackboard Architectures
- Blind spots
- The judge may not recognize sophisticated reasoning it couldn't produce itself
- Lesson 809 — Choosing the Judge Model
- break-even point
- The break-even point is where total API costs equal total self-hosting costs.
- Lesson 122 — API vs Self-Hosted Break-Even Analysis
- Buffering
- Buffering means temporarily holding received tokens in memory before displaying them.
- Lesson 113 — Buffering and Display Strategies
- Built-in Observability
- Every task execution is logged with inputs, outputs, duration, and errors. You can visualize your entire pipeline flow in Prefect's UI, making debugging much easier than scattered print statements.
- Lesson 1799 — Prefect for LLM Pipelines
C
- Cache Hit Rate
- Your cache hit rate is the percentage of requests served from cache versus those requiring fresh API calls:
- Lesson 961 — Monitoring Cache Hit RatesLesson 1166 — Measuring Cache Hit Rates and Parallel Gains
- Cache invalidation
- Cache invalidation is the active removal of specific cache entries when you know they're no longer valid—for example, when source documents are updated or user preferences change.
- Lesson 1159 — Cache Invalidation and TTL Strategies
- Cached Responses
- For common queries, return previously cached results. A slightly stale answer is often better than no answer, especially in RAG systems where document content changes slowly.
- Lesson 1794 — Fallback Strategies and Graceful Degradation
- Caching
- Caching means storing frequently accessed state in fast memory (like Redis or in-process memory) rather than hitting the database repeatedly.
- Lesson 724 — Performance Optimization for State Access
- Calibration
- Calibration is closely related: it means that when your model says "70% confident," it should actually be right 70% of the time — and this should hold consistently across groups.
- Lesson 1568 — Predictive Parity and Calibration
- Calibration Sessions
- Calibration Sessions are regular check-ins where:
- Lesson 843 — Annotator Training and Calibration
- canary deployment
- A canary deployment is similar—you deploy to a small "canary" group first and watch for problems before full release.
- Lesson 1864 — Gradual Rollouts and Canary Deployments
- canary deployments
- Shadow testing and canary deployments are two strategies that reduce risk:
- Lesson 836 — Shadow Testing and Canary Deployments
- canary release
- A canary release is like sending a canary into a coal mine before miners enter—you test something risky on a small scale first.
- Lesson 916 — Canary Releases and Progressive Rollouts
- Capacity-based limits
- Set hard caps (e.g., store only the 1000 most recent memories, or 500KB of vector embeddings). When limits are reached, remove the oldest or lowest-scoring entries.
- Lesson 604 — Forgetting and Memory Pruning
- Centralized log aggregation
- Centralized log aggregation means routing all logs from every component to a single platform where you can search, filter, and analyze them together.
- Lesson 1509 — Centralized Log Aggregation
- chain
- A chain is a sequence of operations that processes data step-by-step.
- Lesson 505 — Chains: The Core Abstraction
- Chain-of-Thought
- Chain-of-Thought is about thinking out loud.
- Lesson 181 — ReAct vs Chain-of-Thought Differences
- Challenges
- Sentences vary wildly in length—one might be 5 words, another 50. Some "sentences" in technical docs or legal text are massive. You need reliable sentence boundary detection (trickier than it sounds with abbreviations like "Dr." or "Inc.").
- Lesson 338 — Sentence-Based Chunking
- character encoding
- This system is called character encoding.
- Lesson 470 — Character Encoding and Unicode Handling
- Chatbots and conversational interfaces
- Chatbots and conversational interfaces are prime candidates.
- Lesson 932 — When to Use Synchronous Patterns
- checkpoint
- A checkpoint is a saved snapshot of a model at a specific point in its training.
- Lesson 45 — Model Variants and Checkpoints
- Checkpointable state
- The entire graph state can be serialized, enabling resumable workflows
- Lesson 706 — LangGraph for Multi-Agent State Management
- Checkpointing
- Checkpointing is the practice of creating these snapshots at strategic points: before expensive operations, after successful tool calls, or at fixed intervals.
- Lesson 485 — Progress Tracking and CheckpointingLesson 621 — State Serialization and CheckpointingLesson 1771 — Intermediate Result Storage and CheckpointingLesson 1804 — Checkpointing and Recovery Patterns
- Chroma
- Chroma is the simplest to start with—think of it as the "SQLite of vector databases." It's embedded by default (runs in your Python process), perfect for prototyping and small-scale applications.
- Lesson 305 — Open Source Vector DB Landscape
- Chunked Transfer Encoding
- Chunked Transfer Encoding is an HTTP mechanism that lets your server send data in pieces (chunks) without declaring a Content-Length header beforehand.
- Lesson 996 — Chunked Transfer Encoding
- Circuit breakers
- Circuit breakers are monitoring patterns that detect failures and stop sending traffic to a failing component.
- Lesson 918 — Rollback Strategies and Circuit Breakers
- Citation quality metrics
- Citation quality metrics are standardized measurements that help you assess whether your system is attributing information correctly, covering all sources it should, and only citing relevant material.
- Lesson 368 — Citation Quality Metrics
- Class distribution
- Monitor which categories are being predicted. Sudden changes might indicate new data patterns.
- Lesson 1659 — Monitoring Vision Model Performance
- Classification
- Classification means bucketing errors into types:
- Lesson 1792 — Error Detection and Classification
- Classification tasks
- Sentiment analysis or topic categorization are direct pattern matches
- Lesson 171 — When CoT Helps vs When It Doesn't
- Closing the loop
- Closing the loop means demonstrating that their input mattered, which encourages continued engagement and builds trust.
- Lesson 1405 — Closing the Loop with Users
- Coherence
- The bot needs to remember what the user just said to respond appropriately.
- Lesson 735 — Conversation Context Fundamentals
- collection
- In Chroma, a collection is like a table in a traditional database—it's a named container that holds your vectors, documents, and associated metadata.
- Lesson 307 — Chroma: Collections and MetadataLesson 313 — Milvus: Collections and Indexes
- Color coding
- Different span types (LLM calls, tool usage, chains) are visually distinct
- Lesson 1264 — LangSmith Trace Visualization and Debugging
- Combine signals
- CTR + dwell time + completion is stronger than any single metric
- Lesson 1391 — Signal Extraction from Implicit Feedback
- Commercial use
- Commercial use means anything that generates revenue or supports a business — including internal company tools.
- Lesson 42 — Model Licensing and Usage Rights
- Comparing prompt variations
- Comparing prompt variations means running multiple prompt candidates against the same test suite and evaluating them with:
- Lesson 1170 — Comparing Prompt Variations
- Comprehensive coverage
- A research agent + fact-checker + summarizer together cover more ground than any single agent
- Lesson 690 — Parallel Agent Execution
- Concept Drift
- Concept Drift is the most subtle: the relationship between inputs and correct outputs changes.
- Lesson 1243 — Understanding Distribution Drift in LLM Systems
- Concurrent
- Concurrent means you place all orders at once and collect them as they're ready—much faster.
- Lesson 1162 — Async/Await and Concurrent API Calls
- Conditional availability
- Conditional availability means deciding which groups or individual functions to send to the LLM based on runtime conditions.
- Lesson 563 — Function Grouping and Conditional Availability
- Conditional Routing
- Edges can be conditional functions that inspect state and determine the next node dynamically. For example, after a web search, route to "summarize" if results are good, or "refine_query" if they're insufficient.
- Lesson 1800 — LangGraph for Agent Workflows
- Conflicting constraints
- A field marked both required and with default: null
- Lesson 982 — Validation for Structured Output Requests
- Connection Establishment
- The initial TCP handshake to open a connection to the API server. This happens once per connection but can add 50-200ms depending on geographic distance.
- Lesson 1140 — Network Latency and API Response Times
- Constitutional AI
- Anthropic's Claude takes a fundamentally different approach called Constitutional AI.
- Lesson 86 — Anthropic Claude API: Constitutional AI Approach
- Context
- The LLM still gets enough surrounding text to understand references, pronouns, and flow
- Lesson 389 — Sentence Window Retrieval
- Context Assembly
- Confirm retrieved chunks are properly formatted and passed to the LLM with the right prompt template.
- Lesson 893 — Testing Complete RAG Pipelines
- Context compression on-the-fly
- Context compression on-the-fly means processing retrieved documents after retrieval but before prompt injection to extract only the most relevant parts.
- Lesson 359 — Context Compression On-the-Fly
- Context matters
- A short dwell time on a simple answer is good; on complex content it's bad
- Lesson 1391 — Signal Extraction from Implicit Feedback
- Context relevance instructions
- Context relevance instructions are prompt directives that tell the LLM to actively filter and prioritize the context you've provided.
- Lesson 355 — Context Relevance Instructions
- Context Window Issues
- Truncated responses, ignored instructions buried in long prompts, or confusion when context is too large.
- Lesson 1296 — Analyzing Prompt-Response Pairs
- Continuous ground truth updates
- Continuous ground truth updates means establishing processes to regularly refresh your evaluation datasets so they stay aligned with your system's current challenges.
- Lesson 828 — Continuous Ground Truth Updates
- Continuous red-teaming
- Continuous red-teaming means systematically analyzing production data to discover new vulnerabilities, then feeding those insights back into automated adversarial testing that runs regularly alongside model updates.
- Lesson 1471 — Continuous Red-Teaming in Production
- Control
- Control means you manage everything yourself—choosing exact configurations, customizing behavior, and owning your infrastructure.
- Lesson 24 — Control vs Convenience Trade-offs
- Control blast radius
- If something breaks, only a small percentage is affected
- Lesson 878 — Progressive Rollouts and Feature Flags
- Convenience
- Convenience means letting someone else handle the complexity.
- Lesson 24 — Control vs Convenience Trade-offs
- Conversation context
- Conversation context is the accumulated information from previous exchanges between a user and a chatbot—essentially, the "memory" of what's been discussed so far.
- Lesson 735 — Conversation Context Fundamentals
- Conversation state
- Conversation State is structured, extracted information that represents what the system knows about the conversation.
- Lesson 581 — Limiting Available Tools by ContextLesson 742 — Conversation State vs Message History
- ConversationBufferMemory
- ConversationBufferMemory is LangChain's basic memory component that stores the entire conversation history in a simple buffer (like a list).
- Lesson 509 — Memory: ConversationBufferMemory
- correlation ID
- A correlation ID (or trace ID) is a unique identifier—typically a UUID—generated at the start of each user request and passed along to every downstream operation.
- Lesson 1222 — Correlation IDs and Request Tracing
- Cost attribution by feature
- Cost attribution by feature means labeling each API request with metadata that identifies which part of your application generated it.
- Lesson 1179 — Cost Attribution by Feature
- Cost gates
- Cost gates are automated checks that enforce spending limits before tests run or deployments proceed.
- Lesson 908 — Cost Gates and Budget Limits
- Cost optimization
- If query is simple → use small model, else → use large model
- Lesson 1768 — Branching Logic and Conditional Steps
- cost per interaction
- Understanding your cost per interaction is essential to determining whether your AI product is financially viable at scale.
- Lesson 1854 — Cost per Interaction and Unit Economics
- Crew
- In CrewAI, a crew is a coordinated team of agents working together toward a common objective.
- Lesson 704 — CrewAI Framework FundamentalsLesson 705 — Defining Crews and Assigning Roles in CrewAI
- Cron Schedules
- Cron Schedules are time-based triggers that run pipelines at fixed intervals—daily at 2 AM, every Monday, hourly during business hours.
- Lesson 495 — Scheduling and Triggering Strategies
- Custom formats
- Storing data in a provider-specific vector database schema
- Lesson 22 — Evaluating Vendor Lock-in Risk
D
- Data drift
- The input distributions shift. Users start asking questions in new formats, using different vocabulary, or approaching your system with novel query patterns.
- Lesson 1426 — Detecting and Addressing Model Degradation
- Data pipeline infrastructure
- Data pipeline infrastructure is the plumbing that collects all this chaos and delivers it in a usable form.
- Lesson 16 — Data Pipeline Infrastructure
- Data Transfer
- The actual time to send your prompt and receive the response over the wire. Large prompts or slow connections significantly impact this. Network bandwidth and latency both matter here.
- Lesson 1140 — Network Latency and API Response Times
- Data versioning
- Data versioning is like Git for your datasets—it captures snapshots of your training data at specific points in time.
- Lesson 1322 — Data Versioning and Lineage
- Delimiters
- Delimiters are special characters or strings that mark boundaries in the output.
- Lesson 158 — Delimiters and Markers for Parsing
- deployment pipeline
- A deployment pipeline is an automated sequence of stages that code, prompts, or models pass through before reaching users—think of it as a conveyor belt with quality checkpoints.
- Lesson 920 — Deployment Pipelines and Approval Gates
- Deployments
- Pods are individual cooking stations, Deployments are the head chef's recipe and staffing plan, and Services are the waiters connecting customers to the kitchen.
- Lesson 1102 — Kubernetes Core Concepts: Pods, Deployments, Services
- deprecation
- Meanwhile, deprecation is the gentle, transparent process of saying: "Version 1 still works, but please migrate to Version 2—we'll remove v1 in 6 months."
- Lesson 1002 — Backward Compatibility and Deprecation
- description
- A plain-English explanation of what the function does and when to use it. This is critical—the LLM reads this to decide whether to call your function. Be specific: "Gets current weather for a given city" is better than "Weather function."
- Lesson 555 — Function Schema Structure and OpenAI Format
- Detection
- Detection means instrumenting each step to catch failures.
- Lesson 1792 — Error Detection and Classification
- Deterministic testing
- The same input produces the same behavior
- Lesson 1301 — Reproducing Issues Locally
- Device mapping
- Device mapping is the strategy you use to decide which layers live on which GPU (or CPU) to balance memory usage and maximize throughput.
- Lesson 1077 — Device Mapping Strategies
- Direct Messages
- Direct Messages are private conversations users initiate with your bot.
- Lesson 1821 — Slack Event Handling and Commands
- Discovery
- Discovery is the process by which an agent queries this registry to find the right specialist for a task.
- Lesson 676 — Agent Registry and Discovery
- Distribution
- Distribution means assigning those subtasks to agents based on their specific capabilities and roles.
- Lesson 694 — Task Decomposition and Distribution
- Distribution shift
- The underlying relationship between inputs and outputs changes. What used to be a good answer six months ago may no longer satisfy current user expectations or domain requirements.
- Lesson 1426 — Detecting and Addressing Model Degradation
- Diversity-aware retrieval
- Diversity-aware retrieval means going beyond pure similarity scoring.
- Lesson 1580 — Retrieval Debiasing in RAG Systems
- Document
- A Document is the raw, unprocessed unit of data you feed into LlamaIndex.
- Lesson 514 — Documents and Nodes: LlamaIndex Data Model
- document loader
- A document loader is a standardized interface that converts various file types (PDFs, Word docs, web pages, etc.) into a common format your AI pipeline can process.
- Lesson 465 — Document Loaders and Abstractions
- Documentation
- Documentation is how you preserve what you've learned.
- Lesson 1173 — Iteration Velocity and Documentation
- Domain vocabulary
- The specific jargon, abbreviations, and context your users bring
- Lesson 1387 — The Production Data Advantage
- Dropped Frames
- The count of frames skipped or discarded. High drop rates indicate your pipeline can't keep pace with the input stream.
- Lesson 1670 — Video Inference Monitoring and Debugging
- Dynamic adapter loading
- Dynamic adapter loading means loading adapter weights into memory only when a request requires them, then optionally unloading them to free space for the next adapter.
- Lesson 1371 — Dynamic Adapter Loading
E
- Early stopping
- Early stopping means halting training when validation performance stops improving, even if training loss could go lower.
- Lesson 1331 — Overfitting Detection and Early Stopping
- Edge cases that matter
- The weird, ambiguous, or poorly-formed inputs that happen in practice
- Lesson 1387 — The Production Data Advantage
- Edge computing
- Edge computing means running CV models directly on devices near where data is captured—security cameras, drones, smartphones, IoT sensors—rather than sending data to remote cloud servers.
- Lesson 1671 — Edge Computing Fundamentals for CV
- Empty Citation Check
- If your retrieved context is non-empty but the response contains zero citations, flag this as a potential issue.
- Lesson 367 — Handling Missing or Hallucinated Citations
- Endpointing
- Endpointing is the process of determining when a speaker has completed their utterance and it's time for the system to respond.
- Lesson 1708 — Endpointing and Turn-Taking Detection
- Engineering time
- Engineering time is typically the hidden giant.
- Lesson 1083 — Understanding Total Cost of Ownership for Self-Hosted LLMs
- Enum enforcement
- Restricted choices are guaranteed
- Lesson 760 — Function Calling for Structured Output
- Equal Opportunity
- Equal Opportunity means the model's true positive rate (recall) should be the same for all protected groups.
- Lesson 1567 — Equal Opportunity and Equalized Odds
- Equalized Odds
- Also ensures unqualified applicants are rejected at similar rates across groups
- Lesson 1567 — Equal Opportunity and Equalized Odds
- error state
- An error state is a named state in your machine specifically for handling failures.
- Lesson 1784 — Error States and Recovery Strategies
- Error-free parsing
- The API won't return malformed JSON
- Lesson 760 — Function Calling for Structured Output
- Escaping
- Escaping means converting special characters into safe representations.
- Lesson 154 — Escaping and Sanitizing User Input
- Evaluation and testing frameworks
- Evaluation and testing frameworks are specialized tools designed to assess:
- Lesson 17 — Evaluation and Testing Frameworks
- Exact unlearning
- Exact unlearning means retraining your model from scratch, excluding the requested data entirely.
- Lesson 1549 — Exact Unlearning vs Approximate Unlearning
- Example selection and pruning
- Example selection and pruning means strategically choosing a smaller set of high-quality, diverse examples that teach the pattern without wasting context window space.
- Lesson 1149 — Example Selection and Pruning
- Explicit consent
- Explicit consent is clear, affirmative action: a user clicks "I agree to have my data used for AI training." It's unambiguous and audit-friendly.
- Lesson 1545 — Consent Models for AI Training Data
- Explicit feedback
- Explicit feedback is direct and intentional—users actively tell you what they think.
- Lesson 1397 — Implicit vs Explicit Feedback
- Explicit goal markers
- The agent declares "task complete" in its output
- Lesson 623 — Stopping Conditions: Goal Achievement
- Explicit state representation
- The current node shows exactly which agent is active
- Lesson 706 — LangGraph for Multi-Agent State Management
- Exponential Backoff
- Exponential backoff means waiting progressively longer between retry attempts: first retry after 1 second, then 2 seconds, then 4, then 8, and so on.
- Lesson 494 — Retry Logic and Error HandlingLesson 1793 — Retry Logic and Exponential Backoff
- Extraction
- Extraction means pulling out the individual steps from the model's response.
- Lesson 172 — Extracting and Validating Reasoning Steps
F
- Fail fast and learn quickly
- A bad experiment costs minutes and cents, not days and dollars
- Lesson 1386 — PEFT for Rapid Iteration and Experimentation
- Fallback Mechanisms
- Fallback Mechanisms are your safety nets:
- Lesson 723 — State Recovery and Error Handling
- Fast-path optimization
- the first tier must be genuinely fast, or latency compounds
- Lesson 1200 — Cascade Pattern for Model Routing
- Feature drift
- Feature drift is often the culprit: the statistical properties of your input features have changed, but your model still expects the old patterns.
- Lesson 1628 — Feature Monitoring and Drift Detection
- Feature flags
- Feature flags are the control mechanism that lets you dynamically adjust these percentages without redeploying code.
- Lesson 878 — Progressive Rollouts and Feature Flags
- feature store
- A feature store is a centralized repository that:
- Lesson 1620 — Feature Store Fundamentals
- feature store integration pattern
- The feature store integration pattern is your blueprint for connecting both worlds to the same source of truth—ensuring the feature named user_30d_avg_purchase means exactly the same thing whether it's powering a training job or a real-time prediction.
- Lesson 1635 — Feature Store Integration Patterns
- Fine-tuning
- Fine-tuning means training the model on your specific dataset to internalize patterns and behaviors.
- Lesson 1303 — Fine-Tuning vs Prompt Engineering Trade-offs
- Finite State Machine
- A Finite State Machine consists of four fundamental elements that work together to model behavior:
- Lesson 1778 — Finite State Machines (FSM) Basics
- Fix output length and parameters
- Set a low, fixed max_tokens and temperature=0 so generation time is deterministic and consistent.
- Lesson 537 — Performance Comparison: Framework vs Raw
- Fixed iteration count
- Unlike text generation where sequences finish at different times, diffusion steps are predictable
- Lesson 1028 — Batching for Different Model Architectures
- Fixed-size chunking
- Fixed-size chunking is the simplest strategy: you divide text into uniform segments of N characters or tokens, optionally with overlap between consecutive chunks.
- Lesson 336 — Fixed-Size Chunking
- fixture
- In traditional software, a fixture is reusable setup code or data that multiple tests share.
- Lesson 890 — Test Coverage and Fixtures for AI Systems
- Flagging
- Flagging is safer when you're uncertain—route borderline cases to human reviewers rather than auto-correcting and potentially changing intended meaning.
- Lesson 1585 — Output Filtering and Rewriting
- Flash Attention
- Flash Attention is a hardware-aware algorithm that restructures attention computation to minimize memory reads/writes.
- Lesson 1036 — Flash Attention and Kernel Optimizations
- Flows
- Flows are the top-level containers for your pipeline logic—think of them as the "job" you want to run (like "update vector database" or "batch embed documents").
- Lesson 491 — Prefect for Modern AI Workflows
- FP16
- FP16 is common on NVIDIA GPUs, while BF16 (bfloat16) is newer and handles extreme values better, making it safer for many models without careful tuning.
- Lesson 70 — Mixed Precision Inference
- Framework Benefits
- These integrations eliminate boilerplate. You define the schema once in Pydantic, and the framework ensures the LLM's output conforms to it. Validation errors are caught automatically, and you receive type-safe Python objects.
- Lesson 776 — Integration with LLM Frameworks
- Function Calling
- Function Calling (covered earlier) is more direct.
- Lesson 648 — Comparing ReAct to Other Agent Patterns
- Function grouping
- Function grouping means organizing related functions together (e.g., "database_tools", "email_tools", "admin_tools").
- Lesson 563 — Function Grouping and Conditional Availability
G
- Gemini
- Gemini is Google's newest multimodal model family (text, images, video, audio).
- Lesson 1119 — Google Vertex AI Foundation Models
- General principle
- The tokenizer breaks text into the same pieces the model will see
- Lesson 118 — Token Counting and Cost Estimation
- Gold standards
- Gold standards are questions or tasks where you already know the correct answer.
- Lesson 845 — Quality Control and Gold Standards
- golden dataset
- A golden dataset is your curated, trusted collection of test cases with known correct outputs.
- Lesson 824 — Golden Datasets and Versioning
- GPU Auto-Scaling
- Monitor queue depth and spin up/down GPU instances dynamically. Image generation has perfect "embarrassingly parallel" workloads—each request is independent.
- Lesson 1744 — Production Image Generation Pipelines
- GPU utilization
- Percentage of GPU compute actually being used. Low utilization (<50%) suggests CPU bottlenecks or inefficient batching. Maxed-out GPU (>95%) means you're compute-bound.
- Lesson 1659 — Monitoring Vision Model PerformanceLesson 1670 — Video Inference Monitoring and Debugging
- Graceful Degradation
- The main workflow continues while problems are queued
- Lesson 723 — State Recovery and Error HandlingLesson 1796 — Dead Letter Queues and Manual Investigation
- Ground truth
- Ground truth is a collection of examples where you already know the correct answer.
- Lesson 819 — What is Ground Truth and Why It Matters
- guard
- In a state machine, a guard is a boolean condition that must be satisfied before a transition fires.
- Lesson 1782 — Guards and Conditional Transitions
- Guardrail metrics
- Guardrail metrics are your safety net—predefined thresholds that trigger automatic experiment termination when violated.
- Lesson 876 — Guardrail Metrics and Early Stopping
H
- Hallucinations
- The model confidently stating false information or inventing details that weren't in your context. Look for citations to non-existent sources or facts that contradict your retrieved documents.
- Lesson 1296 — Analyzing Prompt-Response Pairs
- handler
- The handler is where your preprocessing pipeline lives.
- Lesson 1650 — TorchServe for Vision Models
- Held-out test sets
- Held-out test sets are your first line of defense.
- Lesson 243 — Evaluating Fine-tuned Embeddings
- Helicone
- Helicone is a proxy-based logging platform.
- Lesson 1282 — Comparing Arize and Helicone Use Cases
- hierarchical organization
- In multi-agent systems, hierarchical organization means arranging agents in layers, similar to a corporate org chart.
- Lesson 691 — Hierarchical Agent Organization
- HLS and DASH
- HLS and DASH are adaptive streaming protocols—better for recorded content than live interaction due to 2-10 second latencies, but useful when you need broad device compatibility.
- Lesson 1669 — WebRTC and Low-Latency Streaming Protocols
- HMAC Signature Verification
- HMAC Signature Verification is your primary defense.
- Lesson 1831 — Webhook Security and Signature Verification
- Human-in-the-Loop Evaluation
- Human-in-the-Loop Evaluation means involving real people to review your agent's decisions, tool selections, and reasoning chains—especially for complex or high-stakes tasks.
- Lesson 667 — Human-in-the-Loop Evaluation
I
- ID
- A unique string identifier for each vector
- Lesson 298 — Upserting Vectors to PineconeLesson 299 — Querying and Filtering in Pinecone
- Idle resources
- Idle resources are any cloud assets consuming money without providing value: stopped instances still attached to storage, orphaned disk volumes from deleted VMs, elastic IPs without attached instances, or load balancers pointing to nothing.
- Lesson 1217 — Idle Resource Detection and Cleanup
- If the guard returns false
- transition is blocked; the system stays in the current state
- Lesson 1782 — Guards and Conditional Transitions
- image
- An image is a blueprint—a read-only template containing your Python environment, model weights, dependencies, and code.
- Lesson 1092 — Docker Basics for AI Engineers
- Immediately after successful task completion
- Immediately after successful task completion is your golden window.
- Lesson 1399 — Timing and Context for Feedback Requests
- Implicit feedback
- Implicit feedback is indirect and behavioral—you infer quality from what users do, not what they say.
- Lesson 1397 — Implicit vs Explicit Feedback
- Impractical at scale
- Handling thousands of deletion requests individually is impossible
- Lesson 1548 — Machine Unlearning Fundamentals
- In-memory state storage
- In-memory state storage means keeping this information in your application's RAM using simple data structures like Python dictionaries.
- Lesson 716 — In-Memory State Storage
- Include full state
- A complete checkpoint contains model weights, optimizer state, scheduler state, current epoch/step number, and training configuration. This ensures perfect resumability.
- Lesson 1329 — Checkpoint Management and Recovery
- Incomplete Responses
- Answer is technically correct but unhelpful
- Lesson 1872 — Identifying Failure Modes Through User Feedback
- Inconsistency
- The judge may lack the nuance to distinguish between subtle quality differences
- Lesson 809 — Choosing the Judge Model
- Indexes
- Indexes are the top-level containers in Pinecone where you store and query vectors.
- Lesson 296 — Pinecone Architecture and Concepts
- Inference costs
- Inference costs are typically your largest expense: LLM API calls (GPT-4, Claude, etc.), embedding generation, and model hosting.
- Lesson 1880 — Cost Structure Analysis and Margin Calculation
- Inference optimization
- Inference optimization is the practice of making your model's predictions (inferences) faster, more efficient, and more cost-effective when serving real users.
- Lesson 61 — What is Inference Optimization
- Information Extraction
- Beyond narrative descriptions, you can extract structured data: count objects, identify text within images (OCR-like tasks), determine spatial relationships ("the cat is on the table"), or answer specific questions about image content.
- Lesson 1739 — Image Understanding and Captioning
- Input and output snapshots
- The raw user input and generated response (respecting privacy requirements)
- Lesson 1462 — Logging and Audit Trails
- Input examples
- The prompts or queries your system receives
- Lesson 1265 — Creating and Managing Datasets in LangSmith
- Inputs and outputs
- The exact text or data that went in and came out
- Lesson 1264 — LangSmith Trace Visualization and Debugging
- intents
- Unlike Slack's event-driven architecture, Discord bots connect via a persistent WebSocket gateway and use a more granular permissions system called intents.
- Lesson 1822 — Discord Bot Development with LLMs
- Intersectional fairness
- Intersectional fairness means examining AI system performance across combinations of protected attributes simultaneously, not just in isolation.
- Lesson 1573 — Intersectionality and Multi-attribute Fairness
- Iteration velocity
- Iteration velocity is how quickly you can test new ideas.
- Lesson 1173 — Iteration Velocity and Documentation
- Iterative Prompt Refinement
- Iterative Prompt Refinement is the practice of treating prompt engineering like debugging code.
- Lesson 136 — Iterative Prompt Refinement
- Iterative Refinement
- Review disagreements, update guidelines, retrain annotators, and repeat. The goal is consistent, reliable labeling.
- Lesson 821 — Manual Annotation Workflows
J
- JSON mode
- JSON mode is a setting available in modern LLM APIs (like OpenAI's GPT-4, Anthropic's Claude, and others) that guarantees the model's response will be valid JSON.
- Lesson 756 — JSON Mode Basics
K
- Key rotation
- Key rotation means cycling through a pool of API keys automatically.
- Lesson 103 — Multi-Key Rotation Strategies
- knowledge base
- In RAG, the knowledge base is the collection of information your system searches through to find relevant context for the LLM.
- Lesson 329 — The Knowledge Base in RAG
- knowledge graph
- This is called a knowledge graph—a network of nodes (entities) connected by edges (relationships).
- Lesson 601 — Entity Memory and Knowledge Graphs
L
- Lagging indicators
- Lagging indicators are outcome-based measurements you can only collect after something has happened—like monthly user retention, total revenue, or final NPS scores.
- Lesson 1857 — Leading vs Lagging Indicators
- Language Support
- Whisper is your Swiss Army knife for multilingual scenarios. Wav2Vec2 has strong coverage for major languages but may lack models for low-resource languages. Conformer availability depends on the specific implementation and training data.
- Lesson 1713 — ASR Model Landscape and Selection Criteria
- Latency
- Smaller Wav2Vec2 models often have the edge for real-time applications. Whisper's "tiny" and "base" variants balance speed and accuracy. Conformer models vary widely—some are optimized for streaming with low latency, others prioritize batch accuracy.
- Lesson 62 — Measuring Inference PerformanceLesson 262 — Recall vs Latency ConfigurationLesson 411 — Latency and Throughput MetricsLesson 1019 — Batch Size SelectionLesson 1636 — Hybrid Architectures and PrecomputationLesson 1713 — ASR Model Landscape and Selection Criteria
- Latency Targets
- If requests are taking too long, shrink batches to reduce queueing delay—even if it means lower throughput.
- Lesson 1025 — Adaptive Batching Strategies
- Latent failures
- An LLM might slowly drift in quality over days as user queries change, your cached contexts become stale, or model behavior shifts. Traditional alerts won't catch this gradual degradation.
- Lesson 1219 — Why Observability Matters for LLM Systems
- Least privilege
- Least privilege means granting only the minimum permissions needed for each role to perform their job—nothing more.
- Lesson 1521 — Access Controls and Role-Based PermissionsLesson 1843 — Scoped Permissions and Least Privilege
- Length and verbosity control
- Length and verbosity control means explicitly telling the model how much to say: a single sentence, exactly 100 words, three bullet points, or a comprehensive essay.
- Lesson 132 — Length and Verbosity Control
- Limits
- Limits are the maximum resources your pod can consume—like the fire code capacity of that room.
- Lesson 1105 — Resource Requests and Limits for GPU Workloads
- LlamaIndex
- LangChain and LlamaIndex are two popular examples:
- Lesson 13 — Orchestration Frameworks Overview
- LLM-based relevance scoring
- LLM-based relevance scoring means prompting a language model to evaluate whether a retrieved document answers or relates to a given query.
- Lesson 410 — LLM-Based Relevance Scoring
- LocalAI
- LocalAI is that knife—a drop-in replacement for OpenAI's API that runs locally and handles text generation, embeddings, image generation, audio transcription, and more, all through familiar endpoints.
- Lesson 1055 — LocalAI: Multi-Model Local Serving
- Logging
- Every prompt, completion, token count, and metadata is captured
- Lesson 1016 — Production Deployment ChecklistLesson 1277 — Introduction to Helicone for LLM Observability
- Logit biasing
- Logit biasing means adjusting these scores before selection, making certain tokens more or less likely.
- Lesson 779 — Logit Biasing and Token Masking
- Long-running tasks
- A document processing pipeline with OCR, embedding, and summarization can run for hours without losing progress
- Lesson 1798 — Temporal for AI Workflows
- Long-term memory integration
- Long-term memory integration means connecting your chatbot to persistent storage systems like vector databases or knowledge bases so it can recall past interactions, user preferences, and learned facts across multiple sessions.
- Lesson 744 — Long-Term Memory Integration
- loss function
- Your loss function is your model's north star—it optimizes ruthlessly for whatever you measure.
- Lesson 1557 — Sources of Bias: Model Architecture and Objectives
- Low value
- The scenario is unrealistic or extremely rare
- Lesson 838 — Maintaining and Evolving Your Regression Suite
- Lower latency
- Skip silence means faster end-to-end transcription
- Lesson 1706 — Voice Activity Detection (VAD) in Real-Time
M
- Majority voting
- The simplest approach. If most agents agree on an answer, that becomes the final result. Works best when outputs are discrete (classifications, yes/no decisions).
- Lesson 189 — Majority Voting and Answer AggregationLesson 695 — Result Aggregation Strategies
- malicious intent classifier
- A malicious intent classifier is a dedicated ML model—often a simpler, faster model than your main LLM—trained specifically to recognize patterns associated with adversarial behavior.
- Lesson 1489 — Detecting Malicious Intent with Classifiers
- Managed Endpoints
- Managed Endpoints are the key deployment mechanism.
- Lesson 1117 — Azure Machine Learning for Custom Models
- Maximal Marginal Relevance
- Maximal Marginal Relevance is a re-ranking technique that balances two competing goals:
- Lesson 273 — Diversity and MMR in Search Results
- maximum iteration limit
- Setting a maximum iteration limit is your safety valve.
- Lesson 622 — Stopping Conditions: Max Iterations
- Memory footprint
- A 7B model at FP16 needs ~14GB. Two such models need 28GB+ (plus KV cache overhead). Check if they fit simultaneously or require swapping.
- Lesson 1070 — Multi-Model Serving Considerations
- Message History
- Message History is the raw transcript—a chronological log of every message exchanged between user and assistant.
- Lesson 742 — Conversation State vs Message History
- Message History Formats
- This is why Conversation State vs Message History (lesson 742) and proper Message History Formats (lesson 736) are foundational—they give the model the raw material needed for resolution.
- Lesson 743 — Reference Resolution Across Turns
- Message passing
- Message passing is the mechanism that enables this communication.
- Lesson 679 — Message Passing Between Agents
- Metadata
- Metadata are key-value pairs attached to each vector that describe non-semantic attributes (like {"author": "Alice", "year": 2023}).
- Lesson 307 — Chroma: Collections and Metadata
- Metrics
- Instrument your service with Prometheus-compatible metrics. Track request rate, error rate, latency percentiles (p50, p95, p99), GPU utilization, and queue depth. These numbers tell you what's actually happening versus what you think is happening.
- Lesson 1016 — Production Deployment Checklist
- Milvus
- Milvus is the enterprise-grade heavyweight, built for billion-scale vector search.
- Lesson 305 — Open Source Vector DB Landscape
- Minimal complexity
- Your system is simple enough that a framework adds unnecessary weight
- Lesson 712 — Framework Selection and Custom Solutions
- Minimizing database queries
- Minimizing database queries means batching operations and avoiding redundant lookups.
- Lesson 724 — Performance Optimization for State Access
- Misaligned objectives
- The model optimizes for measured alignment metrics rather than true human values
- Lesson 1596 — Alignment Tradeoffs and Failure Modes
- Missing Required Fields
- A request arrives without a prompt or model_name that your endpoint expects.
- Lesson 976 — Handling Missing and Invalid Parameters
- Mixed precision
- Mixed precision means using less precise formats:
- Lesson 70 — Mixed Precision Inference
- MLflow Model Registry
- MLflow Model Registry is the industry standard—integrate model logging in training, then promote versions via UI or API.
- Lesson 1610 — Model Registry and Version Management
- model
- The embedding model version (e.g., text-embedding-3-small, text-embedding-3-large, text-embedding-ada-002)
- Lesson 215 — OpenAI Embeddings API Overview
- Model artifacts
- While models themselves can be read-only and easily replicated (just copy files), metadata about which model version is active requires coordination.
- Lesson 911 — Model Versioning FundamentalsLesson 1131 — Data Replication for Multi-Region SystemsLesson 1338 — Model Registry and Version Management
- Model capability gaps
- Model capability gaps are fundamental limitations in what a model can do—like asking a small language model to perform complex multi-step reasoning, or expecting a text-only model to understand images.
- Lesson 1311 — Model Capability Gaps vs Training Needs
- model card
- A model card is like a nutrition label for AI models.
- Lesson 41 — Understanding Model Cards
- Model ID
- The exact model version (e.g., gpt-4-0613 vs gpt-4-turbo-preview)
- Lesson 1400 — Tracking Feedback Metadata
- Model naming
- Models like claude-3-opus, claude-3-sonnet, and claude-3-haiku are organized by capability tier (not incremental versions)
- Lesson 86 — Anthropic Claude API: Constitutional AI Approach
- Model outputs
- The raw response, token count, and latency
- Lesson 873 — Tracking and Logging A/B Test Data
- model registry
- A model registry is a centralized repository that stores model artifacts alongside critical metadata: training data versions, hyperparameters, performance metrics, deployment status, and lineage (which code/data produced this model).
- Lesson 906 — Model Registry IntegrationLesson 1605 — Model Registry PatternsLesson 1610 — Model Registry and Version Management
- Model selection impact
- Model selection impact is huge: GPT-4 might cost 10-30× more than GPT-3.5 per request.
- Lesson 33 — Measuring Cost per Request
- Model serving
- Model serving is the opposite challenge: taking that trained model and making it available for real-time predictions at scale.
- Lesson 1005 — What is Model Serving?
- Monitor dependencies
- Track which features are provider-specific versus industry-standard (like OpenAI-compatible APIs).
- Lesson 1124 — Vendor Lock-in and Migration Strategies
- Multi-adapter benchmarking
- Multi-adapter benchmarking means running controlled experiments on held-out validation or test data across all candidate adapters:
- Lesson 1382 — Multi-Adapter Benchmarking and Selection
- multi-cloud strategy
- A multi-cloud strategy means running AI workloads across multiple providers (AWS, Azure, GCP, specialized AI platforms) to optimize costs and avoid dependency on a single vendor.
- Lesson 1218 — Multi-Cloud and Hybrid Strategies
- Multi-tenant key isolation
- Multi-tenant key isolation means provisioning separate API credentials for each tenant (or environment, or customer tier).
- Lesson 1480 — Multi-Tenant Key Isolation
- Multi-user memory isolation
- Multi-user memory isolation means architecting your memory systems so each user or session has its own protected memory store.
- Lesson 606 — Multi-User Memory Isolation
N
- name
- The function's identifier (like get_weather or search_database). Use snake_case and keep it descriptive but concise.
- Lesson 555 — Function Schema Structure and OpenAI FormatLesson 761 — Defining Function Schemas
- NDCG
- NDCG is sophisticated: it considers how relevant each result is (not just yes/no) and where it appears (position matters).
- Lesson 797 — Retrieval Quality Metrics
- Near-zero waste
- Blocks are only allocated as needed, and unused blocks are immediately available
- Lesson 1035 — PagedAttention and vLLM
- Negative pairs
- Negative pairs are items that should have different embeddings:
- Lesson 240 — Contrastive Learning for Embeddings
- Nested objects
- Nested objects are objects inside objects ({ "user": { "address": { "city": "NYC" } } })
- Lesson 762 — Nested Objects and Arrays
- No API costs
- After downloading the model, generating embeddings is free
- Lesson 217 — Sentence Transformers Library
- No server session storage
- The server doesn't maintain session objects or in-memory state between calls
- Lesson 921 — Understanding Stateless Architecture in LLM Applications
- No user-specific data
- The integration doesn't need to act on behalf of individual users
- Lesson 1845 — API Key vs OAuth: When to Use Each
- Node
- A Node is a chunked, indexed piece of a Document.
- Lesson 514 — Documents and Nodes: LlamaIndex Data Model
- Node affinity
- Node affinity is Kubernetes' way of matching pods to nodes based on labels.
- Lesson 1109 — Node Affinity and GPU Node Pools
- Non-commercial
- Non-commercial means personal projects, academic research, or educational purposes only.
- Lesson 42 — Model Licensing and Usage Rights
- Non-deterministic behavior
- The same prompt can produce different outputs. A "successful" API call (200 status) might still return a hallucinated, incorrect, or unusable response. Traditional success/failure metrics miss this entirely.
- Lesson 1219 — Why Observability Matters for LLM Systems
- Numeric scores
- Numeric scores are continuous values, often 0-100.
- Lesson 812 — Binary vs Scalar Judgments
- NVLink
- NVLink is NVIDIA's high-speed interconnect technology, providing 300-600 GB/s bandwidth between GPUs (10-20× faster than PCIe).
- Lesson 1079 — Communication Overhead and Bandwidth
O
- OAuth
- OAuth is a delegation protocol that lets users grant your app limited access to their resources without sharing credentials.
- Lesson 1845 — API Key vs OAuth: When to Use Each
- Observable state changes
- A specific condition is now true (file exists, query answered, approval received)
- Lesson 623 — Stopping Conditions: Goal Achievement
- Offline capability
- Works without internet once models are cached
- Lesson 217 — Sentence Transformers Library
- ONNX Runtime
- ONNX Runtime is a cross-platform inference engine that runs ONNX (Open Neural Network Exchange) models with high performance, regardless of which framework originally trained the model.
- Lesson 1652 — ONNX Runtime for Cross-Framework DeploymentLesson 1673 — ONNX Runtime for Cross-Platform Deployment
- Out-of-Range Values
- A max_tokens value of -50 or a temperature of 5.0 (when valid range is 0.0-2.0).
- Lesson 976 — Handling Missing and Invalid Parameters
- Overlap
- Overlap means duplicating some content between adjacent chunks.
- Lesson 341 — Overlap Strategies
P
- parallel function calling
- This is called parallel function calling.
- Lesson 551 — Parallel Function Calls
- parameter sweep
- A parameter sweep means running the same prompt multiple times while varying generation parameters like temperature, top_p, or max_tokens.
- Lesson 203 — Temperature and Parameter Sweeps
- parameters
- A JSON Schema object defining the input shape
- Lesson 555 — Function Schema Structure and OpenAI FormatLesson 761 — Defining Function Schemas
- Parsing
- Parsing means extracting citation markers using pattern matching:
- Lesson 365 — Parsing and Validating Citations
- Payload
- The actual content—task parameters, data, observations, or responses.
- Lesson 682 — Message Protocols and Schemas
- Performance constraints
- Framework overhead is unacceptable for your latency or resource budget
- Lesson 712 — Framework Selection and Custom Solutions
- Perplexity
- Measures how "surprised" the model is by the validation data. Lower is better. Think of it as the model's confusion level.
- Lesson 1333 — Evaluation Metrics for Fine-Tuned Models
- Persistence
- Persistence means saving your fully-built index (with embeddings, nodes, and structure) to disk or external storage, then loading it back instantly when needed.
- Lesson 524 — Storage Context and Persistence
- PII detection
- PII detection is the practice of scanning text for structured patterns that identify real individuals, so you can redact, block, or flag outputs before they reach users.
- Lesson 1455 — PII Detection Fundamentals
- Pipeline versioning
- Pipeline versioning means tracking these changes systematically—using Git for code, tagging DAG versions, and maintaining separate environments for development and production.
- Lesson 497 — Pipeline Versioning and Testing
- Plan repair
- Plan repair is more surgical—modifying specific steps in the existing plan while preserving what's still valid.
- Lesson 614 — Replanning and Plan Repair
- Plan verification and validation
- Plan verification and validation means checking the plan's quality before committing to execution.
- Lesson 617 — Plan Verification and Validation
- Pod
- A Pod is the smallest deployable unit in Kubernetes—typically one or more containers running together.
- Lesson 1102 — Kubernetes Core Concepts: Pods, Deployments, Services
- Pods
- Pods are individual cooking stations, Deployments are the head chef's recipe and staffing plan, and Services are the waiters connecting customers to the kitchen.
- Lesson 296 — Pinecone Architecture and ConceptsLesson 1102 — Kubernetes Core Concepts: Pods, Deployments, Services
- Positive pairs
- Positive pairs are items that should have similar embeddings:
- Lesson 240 — Contrastive Learning for Embeddings
- Post-filtering
- Post-filtering means running the full vector search first, then filtering out results that don't match your metadata requirements afterward.
- Lesson 277 — Pre-filtering vs Post-filtering
- PostgreSQL with pgvector
- PostgreSQL with pgvector is an extension that adds vector operations to the world's most popular open-source relational database.
- Lesson 290 — Traditional Databases with Vector Support
- Pre-filtering
- Pre-filtering is faster when metadata drastically reduces your dataset (searching 1,000 items instead of 1,000,000).
- Lesson 277 — Pre-filtering vs Post-filtering
- Pre-load at startup
- Load quantized weights during container initialization, not on first request—cold starts are more expensive with quantized models
- Lesson 1048 — Production Deployment of Quantized Models
- Predictive Parity
- Positive predictions are equally accurate across groups. If your model says "approve," it should be correct 90% of the time regardless of group membership.
- Lesson 1565 — Defining Fairness in AI Systems
- presence penalty
- Frequency penalty and presence penalty are numerical parameters (typically ranging from -2.0 to 2.0) that modify how likely the model is to reuse tokens:
- Lesson 142 — Frequency and Presence Penalties
- Preserve base capabilities
- The base model's general knowledge remains intact
- Lesson 1384 — Domain Adaptation with PEFT
- Pricing iteration
- Pricing iteration means analyzing production metrics like API calls per user, token consumption patterns, feature adoption rates, and cost per interaction to adjust your tiers, limits, and packaging.
- Lesson 1886 — Pricing Iteration Based on Usage Patterns
- primary metric
- Your primary metric is the single success criterion that determines whether variant B wins.
- Lesson 1862 — Metrics Selection for AI A/B Tests
- Primary metrics
- Primary metrics are your north star—the single most important measure of success.
- Lesson 870 — Choosing Metrics for AI A/B Tests
- privacy budget
- A privacy budget is the total amount of epsilon you're willing to "spend" across all queries or operations.
- Lesson 1536 — Epsilon and Privacy Budgets
- production rule
- Think of a CFG as a recipe book where each recipe (called a production rule) tells you how to build valid pieces of text.
- Lesson 778 — Context-Free Grammars (CFG) Basics
- Progress tracking
- Progress tracking means showing real-time updates about how much work is done (like a progress bar: "Processing document 1,247 of 10,000").
- Lesson 485 — Progress Tracking and Checkpointing
- Progressive Generation
- Break input text into natural boundaries (sentence endings, punctuation) and synthesize each segment independently. Start playback as soon as the first chunk is ready.
- Lesson 1709 — Real-Time TTS and Audio Synthesis
- Prometheus
- Prometheus is a monitoring system that scrapes metrics from your application endpoints.
- Lesson 1126 — Custom Metrics and Prometheus for AI Scaling
- Prompt details
- The exact prompt template and variables used
- Lesson 873 — Tracking and Logging A/B Test Data
- prompt injection detection classifier
- A prompt injection detection classifier is a specialized model (often a smaller, faster ML model) trained specifically to recognize patterns that indicate injection attempts.
- Lesson 1447 — Prompt Injection Detection Classifiers
- Prompt stuffing
- Prompt stuffing means directly including all your context in the prompt itself:
- Lesson 328 — RAG vs Prompt Stuffing
- Prompt versioning
- Prompt versioning means treating each prompt like software code: assign it a version number, track every change, and maintain a history so you can always return to a previous version if needed.
- Lesson 202 — Prompt Versioning and Change Management
- Prompts and completions
- The exact input text and generated outputs for every request
- Lesson 1267 — Weights & Biases for LLM Tracking
- Prosody
- Prosody refers to the rhythm, stress, and intonation of speech.
- Lesson 1719 — Emotion and Prosody Analysis
- prototyping architecture
- Similarly, prototyping architecture means starting with managed APIs (like OpenAI, Anthropic, or Cohere) that let you build and iterate in days, not months.
- Lesson 29 — Prototyping vs Production Architecture
- Pydantic
- Pydantic is a Python library that solves this through data validation using Python type hints.
- Lesson 765 — Pydantic Basics for LLM Output
- PyPDF2
- PyPDF2 is lightweight and fast, ideal for simple text extraction and reading metadata (author, creation date, page count).
- Lesson 457 — PDF Extraction Fundamentals
- PySyft
- PySyft is the powerhouse for federated learning, enabling you to simulate multi-party computation, secure aggregation, and encrypted training across distributed datasets without centralizing data.
- Lesson 1544 — Practical Tools and Frameworks
Q
- Quality Problems
- Responses that are off-topic, too verbose, poorly formatted, or miss key information from the prompt. Compare what you asked for versus what you got.
- Lesson 1296 — Analyzing Prompt-Response Pairs
- Quantitative constraints
- Quantitative constraints are your primary levers:
- Lesson 1881 — Free Tier and Freemium Strategy
- Query classification and routing
- Query classification and routing means analyzing the user's question before retrieval, categorizing it by type, and then directing it to the most appropriate retrieval strategy.
- Lesson 375 — Query Classification and Routing
- Query rewriting based on previous results
- Query rewriting based on previous results means looking at what the LLM produced, identifying what's missing or unclear, and reformulating your query to retrieve additional context that addresses those gaps.
- Lesson 440 — Query Rewriting Based on Previous Results
- Queue depth
- If requests are backing up in your queue, you need more replicas
- Lesson 1125 — Horizontal Pod Autoscaling for AI Workloads
R
- RAG
- RAG means indexing those documents, retrieving only the relevant ones, then including just those in your prompt.
- Lesson 328 — RAG vs Prompt Stuffing
- RBAC for agents
- RBAC for agents means defining explicit permissions that map each agent's role to:
- Lesson 677 — Role-Based Access Control for Agents
- Read-only by default
- Functions should only retrieve data unless write access is absolutely necessary
- Lesson 1450 — Sandboxing and Least Privilege for Tools
- Readiness probe
- Checks if your model is loaded and can handle requests (e.g., model warm-up complete)
- Lesson 1618 — Health Checks and Graceful Shutdown
- reading traces
- Once your observability tools are collecting data, the real work begins: reading traces means analyzing the captured execution flow to understand what's happening inside your LLM system.
- Lesson 1293 — Reading LLM Traces in Production
- Reference Counting
- Track how many active requests are using each adapter to avoid evicting one that's currently in use.
- Lesson 1376 — Adapter Caching and Warm-Up
- Refusal behavior
- Refusal behavior is how your model says "no" to harmful requests—but the challenge is ensuring it doesn't refuse too much (becoming unusable) or too little (becoming unsafe).
- Lesson 1468 — Evaluating Refusal Behavior
- regression suite
- A regression suite is a version-controlled collection of test cases that you run repeatedly to ensure your LLM system doesn't get worse over time.
- Lesson 829 — What is a Regression Suite for LLM Systems
- Regression testing
- Regression testing means re-running a suite of test cases after every change to ensure old capabilities still work.
- Lesson 668 — Regression Testing and Agent Versioning
- Relationships to nearby words
- The embedding changes based on what's around it
- Lesson 210 — Contextual vs Static Embeddings
- relevance
- In RAG systems, relevance means how well a retrieved chunk actually answers or supports the user's query.
- Lesson 423 — Understanding Relevance in RAG Context
- Relevance scoring
- Track how often each memory is retrieved or referenced. Memories never accessed after initial storage score low and become pruning candidates. This works like cache eviction policies in computer systems.
- Lesson 604 — Forgetting and Memory Pruning
- Replanning
- Replanning means generating an entirely new plan when the current one becomes unworkable.
- Lesson 614 — Replanning and Plan Repair
- Reproducibility
- The exact same model weights and behavior everywhere
- Lesson 1597 — Understanding Model Serialization
- Request Isolation
- Even when batching requests across adapters (as we learned previously), ensure logs, metrics, and error traces are partitioned by tenant. A bug in Customer A's adapter shouldn't expose Customer B's prompts or outputs.
- Lesson 1375 — Multi-Tenant Adapter Serving
- Requests
- Requests are what Kubernetes uses to decide which node can host your pod—it's like reserving a hotel room.
- Lesson 1105 — Resource Requests and Limits for GPU Workloads
- Requirements changed
- The behavior being tested is no longer desired
- Lesson 838 — Maintaining and Evolving Your Regression Suite
- Reranking
- Reranking is your second pass—a more computationally expensive but precise model that rescores the top candidates.
- Lesson 1762 — Multimodal Reranking Strategies
- Resource Efficiency
- AI models are resource-hungry. Kubernetes intelligently schedules containers onto nodes with available GPU resources, maximizes hardware utilization, and prevents resource conflicts between multiple models.
- Lesson 1101 — What is Kubernetes and Why for AI?
- RestrictedPython
- RestrictedPython is Python's answer to safe code execution.
- Lesson 1499 — Language-Specific Sandbox Tools
- Retrieval Quality
- Given a specific query, assert that the correct documents (or chunks) are retrieved. Check both relevance and ranking.
- Lesson 893 — Testing Complete RAG Pipelines
- Retry Strategies
- Some failures are transient (network hiccups, temporary file locks). Implement exponential backoff for retries—wait longer between each attempt, with a maximum retry limit.
- Lesson 476 — Error Handling and Logging in Parsers
- Reusable patterns
- Common patterns like RAG, prompt chaining, and agent loops are pre-built.
- Lesson 499 — What is LangChain and Why Use It
- reward hacking
- Models can become extremely creative at gaming the reward function in ways you never intended—a phenomenon called reward hacking.
- Lesson 1417 — RLHF Safety and Alignment
- RGB vs BGR
- OpenCV loads images in BGR by default, but most deep learning frameworks expect RGB. The channels are simply reversed: [Blue, Green, Red] becomes [Red, Green, Blue].
- Lesson 1641 — Color Space Conversions
- Rollback strategies
- Before deploying, define how you'll undo it. Keep the previous model version warm in a blue-green setup, or use canary deployments with automatic rollback triggers. Document the rollback procedure—when stakes are high, you won't remember the commands.
- Lesson 918 — Rollback Strategies and Circuit BreakersLesson 1016 — Production Deployment Checklist
- rubric
- A rubric is a structured scoring guide that breaks down "quality" into specific, measurable dimensions.
- Lesson 840 — Designing Evaluation Rubrics
S
- Sampling
- Sampling is the solution: systematically capturing only a representative subset of your telemetry data while maintaining statistical confidence in your insights.
- Lesson 1228 — Sampling Strategies for High-Volume SystemsLesson 1288 — Sampling Strategies for High-Volume Systems
- sandbox
- A sandbox is an isolated execution environment where code runs with restricted access to system resources.
- Lesson 1495 — Why Sandboxing for Code Generation
- Sandboxing
- Sandboxing means creating an isolated, restricted environment where code runs with limited permissions.
- Lesson 652 — Sandboxing Python Code Execution
- Sanitizing
- Sanitizing means removing or replacing dangerous content entirely.
- Lesson 154 — Escaping and Sanitizing User Input
- Scalability matters
- Adding new capabilities means adding new agents, not rebuilding one massive system
- Lesson 669 — Introduction to Multi-Agent Systems
- schema
- A schema is the formal specification that validates messages, ensuring they contain the right information in the right shape.
- Lesson 682 — Message Protocols and Schemas
- Schema validation
- Rules that check whether a message is well-formed before processing. Think of it like a bouncer checking IDs—malformed messages get rejected early, preventing downstream errors.
- Lesson 682 — Message Protocols and Schemas
- Schema versioning
- Schema versioning means explicitly tracking different versions of your data structure, like software releases.
- Lesson 790 — Schema Evolution and Versioning
- Secret management services
- Secret management services are purpose-built systems that centralize, encrypt, rotate, and audit access to sensitive credentials.
- Lesson 1475 — Secret Management Services
- seed
- A seed is a number that initializes the random number generator used during text generation.
- Lesson 143 — Seed for Reproducible Generation
- Segmenting feedback by user cohorts
- Segmenting feedback by user cohorts means grouping feedback based on shared user characteristics—experience level, use case, subscription tier, industry, or engagement patterns—then analyzing each group separately to find segment-specific issues and opportunities.
- Lesson 865 — Segmenting Feedback by User Cohorts
- Seldon Core
- Seldon Core is Kubernetes-native and framework-agnostic.
- Lesson 1607 — Serving Frameworks Overview
- Selective retention
- Selective retention means keeping critical messages (like system prompts, key user preferences, or important facts) while removing less relevant turns.
- Lesson 740 — Selective Message Retention Strategies
- sentinel value
- A sentinel value is a special marker in your stream that signals an error has occurred.
- Lesson 999 — Error Handling in Streaming Responses
- Separation of duties
- Separation of duties means the people operating the AI system shouldn't be the same ones auditing it.
- Lesson 1513 — Access Control for Audit Logs
- Sequential
- Sequential means you wait for each person's drink before ordering the next one.
- Lesson 1162 — Async/Await and Concurrent API Calls
- Serialization
- Serialization is the process of converting your validated Pydantic model back into JSON—the format LLMs understand.
- Lesson 774 — Model Configuration and Serialization
- server-side
- The server-side (your backend) is the restricted back office where sensitive operations happen.
- Lesson 1478 — Client-Side vs Server-Side Keys
- Services
- Pods are individual cooking stations, Deployments are the head chef's recipe and staffing plan, and Services are the waiters connecting customers to the kitchen.
- Lesson 1102 — Kubernetes Core Concepts: Pods, Deployments, Services
- Session identity
- Session identity is the unique identifier (like a session ID) that labels one user's conversation thread.
- Lesson 715 — Session Identity and User Tracking
- Session storage
- Session storage means persisting conversation data beyond the lifetime of a single request.
- Lesson 741 — Session Management and Persistence
- shadow model
- A shadow model is a candidate model that runs in parallel with your production model, receiving the same inputs but with its outputs silently logged instead of served to users.
- Lesson 1614 — A/B Testing with Model Shadows
- Shadow Testing
- Shadow Testing means running the new model version in parallel with your production model.
- Lesson 836 — Shadow Testing and Canary Deployments
- Sharding
- Sharding means splitting your data across multiple separate databases, while partitioning divides data within a single database into smaller, manageable chunks.
- Lesson 950 — Database Sharding and Partitioning Strategies
- signatures
- Instead of writing prompts, you write modular programs using composable building blocks called signatures and modules.
- Lesson 529 — DSPy: Programming LLM Pipelines
- Similarity matching
- Beyond exact matches, consider semantic similarity caching—if two prompts are 95% similar, maybe they deserve the same cached response.
- Lesson 1156 — Prompt-Level Caching Strategies
- similarity score threshold
- A similarity score threshold is a minimum value you require before returning a result to the user.
- Lesson 235 — Similarity Score Thresholds
- Simpler requirements
- The third-party doesn't support OAuth or you need quick prototyping
- Lesson 1845 — API Key vs OAuth: When to Use Each
- Simplified features
- Design features that are fast to compute in real-time from the start
- Lesson 1619 — Feature Engineering vs. Feature Serving
- Single-example validation
- Single-example validation is like tasting one spoonful of soup and declaring the entire pot perfect.
- Lesson 197 — Why Test Prompts: Beyond Intuition
- Skip-frame strategies
- Sometimes processing every 3rd frame is acceptable
- Lesson 1661 — Video Inference vs Single-Image Inference
- Slash Commands
- Slash Commands are user-invoked shortcuts like /summarize or /ask-ai.
- Lesson 1821 — Slack Event Handling and Commands
- Small datasets or infrequent queries
- Sorting is fine and simpler to implement
- Lesson 231 — Top-K Retrieval Implementation
- Small sample challenge
- Intersectional groups are often underrepresented in datasets, making both training and evaluation harder
- Lesson 1563 — Intersectionality and Compounding Bias
- Smaller dimensions
- Smaller dimensions are faster and cheaper but may miss subtle distinctions.
- Lesson 219 — Model Selection Criteria
- Source Panels
- A dedicated sidebar or bottom section listing all cited sources with thumbnails, titles, and links. Keeps the main text clean while centralizing references. Users can browse sources independently.
- Lesson 366 — Citation Display Patterns
- span
- A span is an individual unit of work within a trace.
- Lesson 1223 — Distributed Tracing Fundamentals
- Special tokens
- Special tokens are markers the model needs to understand structure.
- Lesson 52 — Tokenizers: Encoding and Decoding
- Spot instances
- Spot instances are unused cloud capacity offered at 60-90% discounts.
- Lesson 1069 — Cloud GPU Options and Spot Instances
- SQL Generation
- An LLM creates database queries based on natural language requests. A malicious user might prompt: "Show me all users where name is 'admin'--' OR '1'='1". If the LLM naively converts this to SQL, it could expose the entire user table.
- Lesson 1492 — SQL and Code Injection in LLM Contexts
- state dict
- A state dict is a Python dictionary that maps each layer to its parameter tensor.
- Lesson 1602 — PyTorch State Dicts and Checkpoints
- state machine
- A state machine is a computational pattern that models a system as a collection of distinct states with well-defined transitions between them.
- Lesson 1777 — What Are State Machines and Why Use Them in AI?
- State management
- Variables, intermediate results, and context are preserved across failures without manual database writes
- Lesson 489 — Pipeline Orchestration FundamentalsLesson 499 — What is LangChain and Why Use ItLesson 1798 — Temporal for AI Workflows
- State persistence
- State persistence means serializing (saving) your state machine's current state, context data, and transition history to durable storage so you can reload and continue exactly where you left off.
- Lesson 1785 — State Persistence and Resumption
- State pruning
- State pruning is the practice of selectively removing or compressing parts of your agent's accumulated state while preserving what matters most for decision-making.
- Lesson 625 — State Pruning and Memory Management
- Stateless execution
- Stateless execution means no side effects persist between runs.
- Lesson 1497 — Serverless Functions as Sandboxes
- Stateless LLM Layer
- Each API call to your LLM is independent. No conversation history lives in the model service itself. This keeps your backend simple, scalable, and easy to load balance.
- Lesson 928 — Hybrid Architectures: Best of Both Worlds
- Static Asset Caching
- Tokenizer files, configuration JSONs, and other static artifacts get cached at CDN edge nodes. These small files are retrieved in milliseconds rather than seconds.
- Lesson 1132 — Regional Model Caching and CDN Strategies
- Static thresholds
- Static thresholds are fixed values you set based on requirements or experience:
- Lesson 1254 — Threshold-Based Alerting
- Statistical power
- Statistical power is your ability to detect a real performance difference when it exists.
- Lesson 827 — Dataset Size and Statistical Power
- stop sequence
- A stop sequence is a specific string of text that tells the model "stop generating immediately when you encounter this." Think of it like setting a finish line in a race—the moment the model writes that exact sequence, generation halts, and everything up to (but not including) that marker is returned.
- Lesson 141 — Stop Sequences and Early Termination
- Stratified sampling
- Stratified sampling means dividing your data into meaningful groups (strata) and sampling from each group proportionally—or deliberately over-sampling rare but important cases.
- Lesson 823 — Sampling Strategies for Coverage
- Stream metadata headers
- Stream metadata headers are HTTP headers sent at the beginning of a streaming response that carry important context about the request and the AI system serving it.
- Lesson 1004 — Stream Metadata and Version Headers
- Stripping HTML
- Stripping HTML means removing tags while keeping text content.
- Lesson 469 — HTML and Markdown Cleaning
- Structured data extraction
- Structured data extraction means capturing not just content, but its organization: table cells with their headers, document sections with their hierarchy, and metadata like authors or creation dates — all while preserving how these elements relate to each other.
- Lesson 468 — Structured Data Extraction from Documents
- Structured output
- Structured output means forcing the LLM to return data in a predictable format (JSON, XML, etc.) instead of casual prose.
- Lesson 755 — Why Structured Output Matters
- Structuring
- Organizing scattered data into clear fields
- Lesson 587 — Observation Space and Input Processing
- Subjective but pattern-based criteria
- Tasks like tone assessment, coherence checking, or instruction following where patterns are recognizable
- Lesson 808 — When to Use LLM-as-a-Judge
- Subtle style rules
- Subtle style rules are hard to capture in prompts (sentence structure preferences, vocabulary choices)
- Lesson 1308 — Style, Tone, and Format Consistency
- Sudden spikes
- A user making 100x their normal requests, possibly indicating a runaway loop or intentional abuse
- Lesson 1247 — Anomaly Detection in Token Usage Patterns
- Synchronous blocking
- The client waits for the response—no queueing
- Lesson 1634 — Online Serving with REST APIs
- Synchronous execution
- Synchronous execution means calling tools one at a time, waiting for each to complete before starting the next.
- Lesson 592 — Synchronous vs Asynchronous Execution
T
- Tagging for lifecycle tracking
- Tagging for lifecycle tracking is your first defense.
- Lesson 1217 — Idle Resource Detection and Cleanup
- Task
- A specific piece of work that needs completion. Tasks have descriptions, expected outputs, and can be assigned to specific agents. They're like work items in a project management system.
- Lesson 704 — CrewAI Framework Fundamentals
- Task decomposition
- Task decomposition is the process of splitting a complex goal into sequential subtasks the agent can actually accomplish with its available tools.
- Lesson 127 — Task Decomposition and Step-by-Step InstructionsLesson 609 — Task Decomposition FundamentalsLesson 694 — Task Decomposition and Distribution
- Tasks
- Tasks are the individual operations within a flow—chunking documents, calling an embedding API, inserting vectors.
- Lesson 491 — Prefect for Modern AI Workflows
- TensorFlow Lite
- TensorFlow Lite is the streamlined version designed specifically for these constrained environments, trading some flexibility for dramatically reduced size and faster inference.
- Lesson 1676 — TensorFlow Lite for Mobile and Embedded
- TensorFlow Serving
- TensorFlow Serving is Google's battle-tested framework designed specifically for TensorFlow models (SavedModel format).
- Lesson 1015 — Framework ComparisonLesson 1607 — Serving Frameworks OverviewLesson 1651 — TensorFlow Serving for Vision
- TensorRT
- After converting your model to ONNX format (from the previous lesson), TensorRT is NVIDIA's high-performance deep learning inference optimizer specifically designed for their GPU hardware.
- Lesson 1674 — TensorRT for NVIDIA Hardware
- Test Flakiness Detection
- Flag tests that intermittently fail. For AI systems, distinguish between acceptable model variance and genuine flakiness. A RAG test that occasionally retrieves slightly different documents might be expected; one that randomly times out is not.
- Lesson 910 — CI Monitoring and Debugging Failures
- Testing with mock data
- Testing with mock data means creating fake but realistic sample variables, rendering your template with them, and checking that the final prompt looks right.
- Lesson 156 — Testing Templates with Mock Data
- text-embedding-ada-002
- The previous standard model producing 1,536-dimensional vectors
- Lesson 215 — OpenAI Embeddings API Overview
- Think of it like
- An automated content reviewer that flags images just as a human moderator would, but instantly and at scale.
- Lesson 1743 — Safety and Content Filtering for Images
- Think of it like plumbing
- Each component (prompt template, LLM, output parser) is a pipe segment. The | operator connects them, and data flows through automatically from left to right.
- Lesson 507 — LCEL: LangChain Expression Language
- Think of it like this
- Training data is like ingredients in a recipe. You need to check if each ingredient is safe, legally sourced, and allowed for your intended dish—you can't "un-bake" problematic ingredients later.
- Lesson 1324 — Data Privacy and Licensing
- Third-party audits
- Third-party audits are structured engagements where you hire specialized security firms to systematically probe your LLM application for vulnerabilities—prompt injections, content filter bypasses, PII leakage, jailbreaks, and more.
- Lesson 1472 — Third-Party Security Audits and Bug Bounties
- threshold
- A threshold is the cutoff point where you decide to take action.
- Lesson 1433 — Confidence Scores and Thresholding
- Throughput
- Throughput is the number of retrieval requests your system can process per unit of time—typically queries per second (QPS).
- Lesson 411 — Latency and Throughput Metrics
- Tie-breaking
- Allow the judge to declare ties when outputs are equally good
- Lesson 813 — Comparative Evaluation (Pairwise)
- Tiered storage
- Tiered storage means matching data access patterns to storage types.
- Lesson 952 — Storage Cost Optimization and Data Lifecycle
- Time-of-day irregularities
- Heavy usage at 3 AM when your users are typically asleep
- Lesson 1247 — Anomaly Detection in Token Usage Patterns
- Timeouts
- Timeouts are your first line of defense.
- Lesson 940 — Timeout and Cancellation HandlingLesson 979 — LLM Provider Error Handling and Retries
- TLS Handshake
- The secure encryption negotiation that follows. This adds another 50-100ms for certificate exchange and cipher agreement. Modern APIs use connection pooling to avoid repeating this for every request.
- Lesson 1140 — Network Latency and API Response Times
- Token budget
- Your context window is finite; examples crowd out actual content
- Lesson 1307 — Latency and Token Budget Constraints
- Token budget allocation
- Token budget allocation means consciously deciding how many tokens to dedicate to:
- Lesson 1153 — Token Budget Allocation
- Tone and Style Guidance
- Tone and Style Guidance means explicitly telling the model how to write, not just what to write.
- Lesson 134 — Tone and Style Guidance
- Tool functions
- The actual callable functions you've defined
- Lesson 589 — Action Space and Tool Calling
- trace
- A trace represents the complete end-to-end journey of a single request through your system.
- Lesson 1223 — Distributed Tracing Fundamentals
- Tracing
- Tracing means capturing a detailed record of every tool call—its name, input arguments, output, and the surrounding context at that moment—so you can replay and debug exactly what happened.
- Lesson 660 — Tracing Tool Calls and ContextLesson 1138 — Tracing Multi-Step LLM Chains
- Training needs
- Training needs are situations where the model could perform the task but needs examples to learn your specific requirements—like adopting your company's writing style, following domain-specific formatting rules, or using specialized terminology correctly.
- Lesson 1311 — Model Capability Gaps vs Training Needs
- Type Constraints
- The field type itself (str, int, bool) is your first filter.
- Lesson 766 — Defining Field Types and Constraints
- Type safety
- Numbers are numbers, strings are strings—no guessing
- Lesson 760 — Function Calling for Structured Output
U
- Uniform Sampling
- Uniform Sampling is the simplest strategy: extract frames at regular intervals (e.g., every 5th frame or 1 frame per second).
- Lesson 1662 — Frame Extraction and Sampling Strategies
- Uniqueness percentage
- Fraction of records that are singletons
- Lesson 1533 — Re-identification Risk Assessment
- Updating Records
- PATCH or PUT requests with the record ID and changed fields. If your AI detects a customer is at risk, it might update an opportunity.stage or add a note to contact.description.
- Lesson 1809 — Reading and Writing CRM Data
- Usage Alerts
- Usage Alerts are notifications triggered when your token consumption or costs exceed predefined thresholds.
- Lesson 1182 — Setting Usage Alerts and Budgets
- Usage Growth
- Visualize active users, request volumes, and adoption rates over time. Highlight which features drive engagement and where growth is accelerating or stalling.
- Lesson 1259 — Executive and Business Dashboards
- User
- Think of it like a play: the system message is the stage directions, the user is one character speaking, and the assistant is another character responding.
- Lesson 91 — System, User, and Assistant Message Roles
- User input
- User input is treated as potentially adversarial content to be processed, not executed
- Lesson 1445 — Instruction Hierarchy and Privilege Separation
- User messages
- User messages are the actual queries or prompts you want answered
- Lesson 91 — System, User, and Assistant Message Roles
- UTF-8
- UTF-8 is the universal translator—it can represent nearly every character from every language.
- Lesson 470 — Character Encoding and Unicode Handling
- Utility loss
- The percentage-point drop in F1, accuracy, or whatever metric matters
- Lesson 1539 — Trade-offs: Privacy vs Accuracy
V
- Validation checks
- The result passes predefined quality or completeness tests
- Lesson 623 — Stopping Conditions: Goal Achievement
- variance
- This variance means you need larger samples to confidently say "Model B is actually better than Model A."
- Lesson 871 — Statistical Power and Sample Size for AI Tests
- Vector
- The numerical embedding (must match your index's dimension)
- Lesson 298 — Upserting Vectors to Pinecone
- Versioned test cases
- A collection of tasks your agent should complete (e.g., "book a flight," "calculate compound interest")
- Lesson 668 — Regression Testing and Agent Versioning
- Visibility
- See which tasks succeeded, failed, or are running
- Lesson 490 — Apache Airflow for AI Pipelines
W
- W&B Tables
- W&B Tables are interactive, spreadsheet-like visualizations that let you organize and compare LLM experiments in a structured format.
- Lesson 1268 — W&B Tables for Prompt Comparison
- Warm-up
- Preload adapters you know will be popular before traffic arrives. This is especially useful after deployment—load your top-5 customer adapters during initialization so the first requests don't experience cold-start delays.
- Lesson 1376 — Adapter Caching and Warm-Up
- Wasted resources
- If sequences are shorter than the max length, unused memory sits idle
- Lesson 1032 — Static vs Dynamic KV Cache Allocation
- Weaviate
- Weaviate is the Swiss Army knife—it's not just a vector database but a full semantic search engine with built-in vectorization modules.
- Lesson 289 — Open Source Vector Databases
- Webhook handlers
- Webhook handlers are HTTP endpoints that receive and validate platform events.
- Lesson 1819 — Communication Platform Bot Fundamentals
- Webhook Reliability
- Communication platforms send HTTP POST requests to your bot's endpoint. If your server is down or responds slowly, the platform may retry (causing duplicate processing) or drop the event entirely. Always:
- Lesson 1827 — Bot Deployment and High Availability
- Window memory
- For reactive tasks where only recent turns are relevant (quick Q&A, transactional bots)
- Lesson 510 — Memory: Summary and Window Memory
- word-level alignment
- This is called word-level alignment or forced alignment, and it's essential for:
- Lesson 1688 — Timestamp and Word-Level Alignment
X
- XState
- XState is the most popular state machine library in the JavaScript/TypeScript ecosystem.
- Lesson 1780 — State Machine Libraries: XState and Python Alternatives
Z
- Zapier
- Zapier is the most user-friendly option with thousands of pre-built app integrations.
- Lesson 1833 — No-Code Platforms Overview