Course contentsShow
← AI Engineering
Lesson 1212 of 1,88629. Cost OptimizationFree lesson

Spot and Preemptible Instances

Leverage spot instances for non-critical workloads and implement fault-tolerant architectures to handle interruptions.

Spot and Preemptible Instances

What You'll Learn

You'll learn how to slash infrastructure costs by using interruptible compute for fault-tolerant AI workloads.

The Core Idea

Spot instances (AWS) and preemptible instances (GCP/Azure) are heavily discounted compute resources—often 60-90% cheaper than on-demand pricing—that cloud providers can reclaim with short notice (typically 30 seconds to 2 minutes). Think of them like standby airline tickets: you get massive savings, but you must be flexible and ready to lose your seat.

For AI engineering, this is transformative for non-critical workloads like:

  • Batch inference jobs
  • Model training experiments
  • Data preprocessing pipelines
  • Development/testing environments

The catch? Your instance can disappear at any moment when the cloud provider needs capacity back.

Architecting for Interruption

To use spot instances effectively, you must design fault-tolerant systems:

Checkpointing

Save training state frequently so you can resume from the last checkpoint rather than starting over.

Job Queues

Use queue-based patterns where interrupted tasks automatically re-enter the queue for retry on another instance.

Stateless Workers

Design inference services that don't hold critical state—if one dies, another handles the next request seamlessly.

Diversification

Request multiple instance types across availability zones to reduce simultaneous interruption risk.

When NOT to Use Spot Instances

Avoid spot instances for:

  • Real-time user-facing inference APIs
  • Critical production services requiring guaranteed uptime
  • Long-running jobs without checkpointing capability

Key Takeaway: Spot and preemptible instances can reduce compute costs by 60-90% for AI workloads, but only when paired with fault-tolerant architectures that gracefully handle interruptions through checkpointing, queuing, and stateless design.