LoadBalancer and Ingress for Model APIs
What you'll learn: How to expose your Kubernetes-deployed model serving endpoints to external users and applications through LoadBalancer services and Ingress controllers.
Why External Access Matters
You've deployed your AI model as a Kubernetes Deployment (lesson 1103), configured it properly (1104), and allocated GPU resources (1105). But right now, it's only accessible inside your cluster. To serve real users, you need to expose it to the internet.
Two Approaches to External Traffic
LoadBalancer Service
Think of this as opening a direct door to your model API. When you create a Service with type: LoadBalancer, your cloud provider provisions an external IP address and routes traffic directly to your Pods.
The analogy: It's like giving your house its own dedicated street address. Simple, but if you have 10 different model APIs, you'll need 10 different addresses (and cloud load balancers aren't free).
apiVersion: v1
kind: Service
metadata:
name: llm-api-service
spec:
type: LoadBalancer
selector:
app: llm-inference
ports:
- port: 80
targetPort: 8000
Ingress Controller
An Ingress acts like a smart routing layer. One external IP address can route to multiple services based on URL paths or hostnames. It's more efficient for production deployments with multiple models.
The analogy: Instead of 10 addresses, you have one main entrance with a receptionist who directs visitors to different departments based on what they ask for (/gpt-model, /stable-diffusion, etc.).
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: model-api-ingress
spec:
rules:
- host: api.yourmodels.com
http:
paths:
- path: /llm
pathType: Prefix
backend:
service:
name: llm-api-service
port:
number: 80
Key Takeaway: LoadBalancer provides simple direct access per service, while Ingress enables cost-effective routing of multiple model APIs through a single entry point with path-based or host-based routing.