Docker with Nvidia card access

travisdh1

I've been working on a bunch of stuff lately, and I'm going to be recording some of the annoyances and fixes in the near future.

For now, it's Docker and getting access to Nvidia cards in containers.

For starters, just use Ubuntu Desktop. I know it sucks running a desktop for servers, but in this case the Desktop installer allows you to install the Nvidia proprietary drivers. I had no luck getting the proprietary drivers working on Ubuntu Server.

Step 1: Install Ubuntu Desktop and make sure to select the proprietary Nvidia driver.
Step 2: Verify nvidia-smi has the correct card(s) listed.
Step 3: Run the script below. Sourced from the Docker and Nvidia sites.

Edit: added nvidia-docker2 to the installed Nvidia software

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings

sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
 $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \

  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \

sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit nvidia-docker2

travisdh1

Next up from the Docker site, how to actually enable the GPU in a Docker Compose file.

Example of a Compose file for running a service with access to 1 GPU device

services:
  test:
    image: nvidia/cuda:12.3.1-base-ubuntu20.04
    command: nvidia-smi
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]