Checking Available Hardware and CUDA Setup
You'll learn how to detect what hardware is available on your system and verify whether PyTorch can use GPU acceleration.
Why This Matters
Before running model inference, you need to know whether your code will run on CPU or GPU. GPUs dramatically speed up inference, but only if CUDA (NVIDIA's parallel computing platform) is properly installed and detected. Think of it like checking if your car has a turbocharger before planning a race—you need to know what you're working with.
Core Concepts
When you load a PyTorch model, it needs to know where to run computations. Your system might have:
- CPU only: Works everywhere but slower for AI workloads
- CUDA-enabled GPU(s): NVIDIA GPUs that support parallel processing
- Multiple GPUs: Enterprise setups with several cards
You must check three things:
- Is CUDA available at all? (PyTorch compiled with CUDA support)
- How many GPU devices can you use?
- What are their properties (memory, compute capability)?
Checking Your Setup
import torch
# Check if CUDA is available
print(f"CUDA available: {torch.cuda.is_available()}")
# Get number of GPUs
print(f"Number of GPUs: {torch.cuda.device_count()}")
# Get current device name
if torch.cuda.is_available():
print(f"Current device: {torch.cuda.get_device_name(0)}")
print(f"Device memory: {torch.cuda.get_device_properties(0).total_memory / 1e9:.2f} GB")
This tells you exactly what hardware you have to work with before placing models on devices.
Key Takeaway: Always check hardware availability first—it determines your inference strategy, batch sizes, and whether you need to adjust your deployment architecture for CPU-only environments.