Course contentsShow
← AI Engineering
Lesson 535 of 1,88613. LLM Orchestration FrameworksFree lesson

Framework vs Raw API Trade-offs

Understanding when frameworks add value versus when direct API calls are simpler and more maintainable.

Framework vs Raw API Trade-offs

When building AI applications, a critical architectural decision is whether to use an orchestration framework (like LangChain) or call the provider's API directly. Understanding this trade-off balances productivity with simplicity and maintainability.

The Raw API: Simplicity and Control

For single-purpose tasks—like text summarization, simple chat, or basic data extraction—the raw API is often the best choice. It minimizes dependencies, reduces the learning curve, and gives you total control over the exact request payload.

Here is a complete, modern OpenAI API call using the instantiated client:

import os
from openai import OpenAI

# Assumes OPENAI_API_KEY is set in the environment
client = OpenAI()

response = client.chat.completions.create(
    model=os.environ["OPENAI_MODEL"],
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain APIs."}
    ]
)

print(response.choices[0].message.content)

Use raw APIs when: You need minimal latency, maximum transparency, minimal dependencies, or have a straightforward request-response workflow.

Frameworks: Abstraction and Orchestration

Frameworks shine when you need to abstract repetitive boilerplate or orchestrate complex, multi-step workflows like Retrieval-Augmented Generation (RAG) or agents. Modern frameworks use composable architectures to handle prompt formatting, output parsing, and tool execution effortlessly.

Here is the equivalent workflow using modern LangChain Expression Language (LCEL):

import os
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

# Assumes OPENAI_API_KEY is set in the environment
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("user", "{input}")
])

model = ChatOpenAI(model=os.environ["OPENAI_MODEL"])

# Compose the chain using the pipe operator
chain = prompt | model
response = chain.invoke({"input": "Explain APIs."})

print(response.content)

Note: Older LangChain abstractions like LLMChain are deprecated and now relegated to langchain-classic. Modern implementations require the declarative composition (prompt | model) seen above.

Use frameworks when: Building complex pipelines, managing conversation memory across sessions, integrating multiple data sources, or if you anticipate frequently swapping LLM providers.

The Hybrid Approach

Many production systems use frameworks for orchestration but drop to raw APIs for performance-critical paths. You can leverage framework components selectively, avoiding abstraction layers where they slow down performance or obscure vital debugging data.

Key takeaway: Frameworks accelerate complex workflows like RAG and multi-agent systems but introduce abstraction overhead; default to raw, instantiated API clients for straightforward tasks to keep your codebase lean and maintainable.