flozi00 Logoflozi00 TechHub
🇬🇧EN🇩🇪DE

Installing NVIDIA Drivers, Docker, and GPU Support on Linux

This guide provides a step-by-step walkthrough for setting up a Linux system (specifically Ubuntu) with NVIDIA drivers, Docker, and the necessary tools to run GPU-accelerated Docker containers. This setup is essential for deep learning, data science, and other workloads that can benefit from NVIDIA GPU power within isolated container environments.

Note: If you are using Ubuntu 24.04 and encounter issues like a black screen or driver installation failures, you may need to apply the nokaslr workaround. See our guide on Solving NVIDIA Driver Issues on Ubuntu 24.04 with nokaslr for a solution.

This article is designed to be easy to follow, even if you have limited experience with Linux. Each command is broken down and explained.

1. Update Your System

Before installing new software, it's always a good practice to update your system's package list. This ensures you have the latest information on available packages and their versions.

sudo apt update

Explanation:

  • sudo: This command stands for "Super User Do" and allows you to run commands with administrative privileges, which is necessary for managing system-wide software.
  • apt update: This command downloads the latest package information from all configured software sources. It doesn't install or upgrade any packages but simply refreshes the local package index.

2. Install NVIDIA Graphics Drivers

To use your NVIDIA GPU, you need to install the proprietary drivers. We will use a trusted Personal Package Archive (PPA) to get up-to-date versions.

Add the Graphics Drivers PPA

sudo add-apt-repository ppa:graphics-drivers/ppa

Explanation:

  • add-apt-repository: This command adds a new software repository to your system's list of sources.
  • ppa:graphics-drivers/ppa: This is the address of the "Graphics Drivers" team's PPA, which provides the latest NVIDIA drivers for Ubuntu.

Update the Package List Again

After adding the new PPA, you need to update the package index again to include the packages from the new source.

sudo apt update

Install the Driver

Now you can install the NVIDIA driver. In this example, we install version 580.

sudo apt install nvidia-driver-580-open -y

Explanation:

  • apt install: The command to install a software package.
  • nvidia-driver-580-open: The name of the driver package. This is an open-kernel (or "open-source") version of the driver.
  • -y: This flag automatically answers "yes" to any confirmation prompts during the installation, making it faster.

After installation, it's a good idea to reboot your system to ensure the new driver is loaded correctly.

sudo reboot

3. Install the NVIDIA CUDA Toolkit

The CUDA Toolkit is a development environment for creating high-performance GPU-accelerated applications. Even if you don't plan to develop applications, many pre-built ones (like those in Docker containers) require CUDA libraries to be present on the host system.

Add the CUDA Repository

First, download and install the NVIDIA CUDA repository keyring to verify the authenticity of the packages.

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb

Explanation:

  • wget: A command-line tool to download files from the internet.
  • dpkg -i: The command to install a Debian package file (.deb). This command installs the keyring, which contains the cryptographic keys used to sign NVIDIA's software packages.

Update and Install CUDA

With the repository configured, update the package list and install the CUDA toolkit.

sudo apt-get update
sudo apt-get -y install cuda-toolkit-13-0

Explanation:

  • cuda-toolkit-13-0: This installs version 13.0 of the CUDA Toolkit.

4. Install Docker Engine

Docker allows you to package applications and their dependencies into isolated environments called containers.

Remove Old Docker Versions

First, remove any old or conflicting Docker packages to ensure a clean installation.

for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done

**Explanation:

  • This command uses a for loop to iterate through a list of older Docker-related package names and removes each one using apt-get remove.

Set Up Docker's apt Repository

Next, set up Docker's official repository to install the latest version of Docker Engine.

# Install packages to allow apt to use a repository over HTTPS
sudo apt-get install ca-certificates curl -y
 
# Create a directory for apt keyrings
sudo install -m 0755 -d /etc/apt/keyrings
 
# Add Docker’s official GPG key
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
 
# Add the Docker repository to Apt sources
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Explanation:

  1. The first command installs prerequisite packages for adding a secure repository.
  2. The next commands download Docker's official GPG key and add it to your system's keyring. This key is used to verify that the Docker packages you download are authentic.
  3. The final echo command adds the official Docker software repository to your system's sources list, so apt knows where to find the Docker packages.

Install Docker Packages

Finally, update the package index and install Docker Engine and its related components.

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

**Explanation:

  • docker-ce: Docker Community Edition, the core engine.
  • docker-ce-cli: The command-line interface (CLI) for interacting with Docker.
  • containerd.io: A container runtime that manages the container lifecycle.
  • docker-buildx-plugin and docker-compose-plugin: Plugins that add extra functionality like building multi-platform images and managing multi-container applications.

5. Enable GPU Support in Docker

To allow Docker containers to access the host's NVIDIA GPU, you need to install the NVIDIA Container Toolkit.

Set Up the NVIDIA Container Toolkit Repository

Similar to Docker, you first need to add the official repository for the NVIDIA Container Toolkit.

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

**Explanation:

  • This command chain downloads the GPG key for the NVIDIA Container Toolkit repository, adds it to your system, and then adds the repository to your sources list.

Install the Toolkit and Configure Docker

Now, install the toolkit package and configure Docker to use the NVIDIA container runtime.

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

**Explanation:

  • nvidia-container-toolkit: This package contains the tools needed to bridge Docker with the host's NVIDIA drivers.
  • nvidia-ctk runtime configure: This command configures the Docker daemon to recognize and use the NVIDIA runtime, which is required for GPU access.
  • systemctl restart docker: This restarts the Docker service to apply the new configuration.

6. Verify the Installation

You can verify that Docker has GPU access by running a sample CUDA container.

docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi

**Explanation:

  • docker run: The command to run a container.
  • --rm: Automatically removes the container when it exits.
  • --gpus all: This crucial flag tells Docker to expose all available host GPUs to the container.
  • nvidia/cuda:12.0.0-base-ubuntu22.04: The name of the Docker image to use. This is an official NVIDIA image with CUDA pre-installed.
  • nvidia-smi: The command to run inside the container. It's the NVIDIA System Management Interface, which displays GPU status.

If the installation was successful, this command will print a table with information about your NVIDIA GPU, just as it would if you ran nvidia-smi directly on your host machine.

Congratulations! You now have a fully functional environment for running GPU-accelerated applications in Docker containers.