Customizing Embedding Models and LLMs
You'll learn how to configure LlamaIndex to use the specific embedding and language models that best fit your use case, rather than relying on defaults.
The Core Idea
By default, LlamaIndex uses OpenAI's models—text-embedding-ada-002 for embeddings and gpt-3.5-turbo for text generation. But your application might need something different: perhaps a local model for privacy, a specialized embedding model for your domain, or a different LLM for cost or performance reasons.
Think of it like switching from a default car engine to one optimized for your terrain—both work, but one performs better for your specific journey.
Configuring Your Models
You customize models through a ServiceContext (in older versions) or global settings. You specify:
- Embedding Model: Controls how your documents and queries become vectors
- LLM: The language model that generates responses
For example, you might use sentence-transformers for local embeddings or Claude for superior reasoning. When you build your index or query engine, LlamaIndex uses these custom models instead of defaults.
Why This Matters
Different models have different strengths:
- Local models keep data private and reduce API costs
- Domain-specific embeddings improve retrieval accuracy in specialized fields
- Alternative LLMs offer better performance, lower cost, or specific capabilities
Without this customization ability, you'd be locked into one provider's ecosystem, unable to optimize for your requirements.
Key Takeaway: Customizing embedding models and LLMs in LlamaIndex lets you choose the right tools for your specific use case—whether that's cost, privacy, performance, or domain expertise—rather than accepting one-size-fits-all defaults.