GPU Support in Docker Containers
You'll learn how to configure Docker containers to access NVIDIA GPUs for PyTorch and TensorFlow workloads, turning your containerized AI apps into GPU-accelerated inference machines.
Why GPU Access Matters in Containers
By default, Docker containers are isolated from hardware—including GPUs. Since deep learning frameworks like PyTorch and TensorFlow depend heavily on CUDA for GPU acceleration, containerized models would run CPU-only without special configuration. That means 10-100x slower inference, which defeats the purpose of the hardware you're paying for.
The NVIDIA Container Toolkit
Think of the NVIDIA Container Toolkit as a bridge that lets Docker containers "see" and use your host's GPUs. It consists of:
- NVIDIA Docker runtime - registers GPUs as available resources
nvidia-container-runtime- hooks into Docker to expose GPU devices- CUDA libraries - bundled in official base images like
nvidia/cuda
Configuration Steps
First, install the NVIDIA Container Toolkit on your host machine (this happens once, outside any container). Then, when running containers, use the --gpus flag:
docker run --gpus all nvidia/cuda:11.8.0-base nvidia-smi
The --gpus all flag tells Docker to expose all available GPUs. You can also specify specific devices: --gpus '"device=0,1"' for GPUs 0 and 1 only.
For PyTorch or TensorFlow, base your Dockerfile on GPU-enabled images and the runtime handles the rest—your Python code using torch.cuda or tf.config.list_physical_devices('GPU') will automatically detect available GPUs.
Key Takeaway: The NVIDIA Container Toolkit and --gpus runtime flag enable seamless GPU access in Docker, letting containerized AI workloads leverage CUDA acceleration just like native applications.