Skip to content
Friendly disclaimer: flozi00 TechHub is a solo side-project next to a full-time job — personal learning notes, no official statements. Verify critical steps yourself.
flozi00 Logoflozi00 TechHub
ENDE

NVIDIA GPU Containers on Ubuntu 24.04

Install a host driver, Docker Engine and the NVIDIA Container Toolkit using their current official instructions, then verify GPU access.

1 min readflozi00
linuxubuntunvidiadockergpuguide

This guide covers Ubuntu 24.04 with an NVIDIA GPU and Docker Engine. Three components have different jobs: the host driver operates the GPU, Docker Engine runs containers, and the NVIDIA Container Toolkit makes the driver and GPU devices available inside them. A full CUDA development toolkit on the host is not required merely to run CUDA-enabled containers; install it separately only if you compile CUDA programs on the host.

1. Install and check the host driver

Follow Ubuntu's NVIDIA driver installation guide, which distinguishes general desktop use from compute/server use and recommends the appropriate package for the detected hardware. Do not assume one fixed driver branch or the open-kernel module works for every NVIDIA GPU. After installing and rebooting, run:

bash
nvidia-smi

If that fails, fix the host driver first. Docker cannot compensate for a driver that does not initialize. On NVSwitch systems, also consult NVIDIA's Fabric Manager guide for matching components and service diagnostics.

2. Install Docker Engine

Use Docker's current Ubuntu installation instructions, including its official apt repository, package names and conflict checks. The repository definition and available package versions can change; follow the maintained instructions rather than copying an old third-party repository recipe. Verify Docker independently:

bash
sudo docker run --rm hello-world

3. Install and configure the NVIDIA Container Toolkit

Follow NVIDIA's Container Toolkit installation guide to add its signed repository and install nvidia-container-toolkit. For a regular, rootful Docker daemon, NVIDIA documents:

bash
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

The first command updates Docker's daemon configuration. Rootless Docker has different configuration steps; do not use the rootful commands for that setup.

Finally, run the sample workload from NVIDIA's documentation:

bash
sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

If this prints the GPU table, the container can access the host's NVIDIA driver. It does not prove that every CUDA application or image is compatible with that driver; check the application's CUDA requirements separately.

For a CUDA initialization failure on a multi-GPU system, see the diagnostic guide.