Course contentsShow
← AI Engineering
Lesson 1106 of 1,88627. Cloud Deployment and ScalingFree lesson

LoadBalancer and Ingress for Model APIs

Exposing model serving endpoints to external traffic using Kubernetes networking primitives.

LoadBalancer and Ingress for Model APIs

What you'll learn: How to expose your Kubernetes-deployed model serving endpoints to external users and applications through LoadBalancer services and Ingress controllers.

Why External Access Matters

You've deployed your AI model as a Kubernetes Deployment (lesson 1103), configured it properly (1104), and allocated GPU resources (1105). But right now, it's only accessible inside your cluster. To serve real users, you need to expose it to the internet.

Two Approaches to External Traffic

LoadBalancer Service

Think of this as opening a direct door to your model API. When you create a Service with type: LoadBalancer, your cloud provider provisions an external IP address and routes traffic directly to your Pods.

The analogy: It's like giving your house its own dedicated street address. Simple, but if you have 10 different model APIs, you'll need 10 different addresses (and cloud load balancers aren't free).

apiVersion: v1
kind: Service
metadata:
  name: llm-api-service
spec:
  type: LoadBalancer
  selector:
    app: llm-inference
  ports:
    - port: 80
      targetPort: 8000

Ingress Controller

An Ingress acts like a smart routing layer. One external IP address can route to multiple services based on URL paths or hostnames. It's more efficient for production deployments with multiple models.

The analogy: Instead of 10 addresses, you have one main entrance with a receptionist who directs visitors to different departments based on what they ask for (/gpt-model, /stable-diffusion, etc.).

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: model-api-ingress
spec:
  rules:
  - host: api.yourmodels.com
    http:
      paths:
      - path: /llm
        pathType: Prefix
        backend:
          service:
            name: llm-api-service
            port:
              number: 80

Key Takeaway: LoadBalancer provides simple direct access per service, while Ingress enables cost-effective routing of multiple model APIs through a single entry point with path-based or host-based routing.