Request-Response Pattern: Synchronous Generation
What you'll learn: How LLM API calls work as simple request-response exchanges, and what to handle when things go wrong.
The Basic Pattern
When you call an LLM API, you're using the most straightforward interaction pattern: synchronous request-response. You send a request (like a prompt), wait for the model to process it, and receive a complete response before your code continues.
Think of it like ordering coffee at a counter: you place your order, wait while the barista makes it, and only leave once you have your drink in hand. Your program "blocks" (waits) until the API returns the full generated text.
How It Works
- You send a request with your prompt and parameters
- The API processes your request (tokenization, generation, detokenization)
- You receive the complete response or an error
- Your code continues with the result
This is different from streaming (which we haven't covered yet) where you get partial responses as they're generated.
Handling the Waiting Game
Timeouts are critical. LLM generation can take seconds or even minutes for long outputs. You must set a maximum wait time to prevent your application from hanging indefinitely if something goes wrong on the API side.
Error responses happen. The API might be overloaded, your request might violate content policies, or you might hit rate limits. Your code needs to check for errors and handle them gracefully instead of crashing.
Key Takeaway: Synchronous generation is the simplest LLM interaction pattern—send a request, wait for the complete response, handle timeouts and errors, then continue with your result.