Course contentsShow
← AI Engineering
Lesson 76 of 1,8862. Working with Pre-trained ModelsFree lesson

Checking Available Hardware and CUDA Setup

Detect available GPUs, check CUDA compatibility, and verify hardware capabilities for model inference.

Checking Available Hardware and CUDA Setup

You'll learn how to detect what hardware is available on your system and verify whether PyTorch can use GPU acceleration.

Why This Matters

Before running model inference, you need to know whether your code will run on CPU or GPU. GPUs dramatically speed up inference, but only if CUDA (NVIDIA's parallel computing platform) is properly installed and detected. Think of it like checking if your car has a turbocharger before planning a race—you need to know what you're working with.

Core Concepts

When you load a PyTorch model, it needs to know where to run computations. Your system might have:

  • CPU only: Works everywhere but slower for AI workloads
  • CUDA-enabled GPU(s): NVIDIA GPUs that support parallel processing
  • Multiple GPUs: Enterprise setups with several cards

You must check three things:

  1. Is CUDA available at all? (PyTorch compiled with CUDA support)
  2. How many GPU devices can you use?
  3. What are their properties (memory, compute capability)?

Checking Your Setup

import torch

# Check if CUDA is available
print(f"CUDA available: {torch.cuda.is_available()}")

# Get number of GPUs
print(f"Number of GPUs: {torch.cuda.device_count()}")

# Get current device name
if torch.cuda.is_available():
    print(f"Current device: {torch.cuda.get_device_name(0)}")
    print(f"Device memory: {torch.cuda.get_device_properties(0).total_memory / 1e9:.2f} GB")

This tells you exactly what hardware you have to work with before placing models on devices.

Key Takeaway: Always check hardware availability first—it determines your inference strategy, batch sizes, and whether you need to adjust your deployment architecture for CPU-only environments.