NVIDIA GPU in Docker With QNAP

When I finally want to get my hands on Stable Diffusion, I realized that I don’t have a powerful enough GPU. However, I do have a QNAP NAS (Network-Attached Storage) that comes with two PCIe Slots, and it was advertised as supporting GPU-passthrough and Docker. By some elementary logic, I deducted that I can just grab a NVIDIA RTX A2000 12GB, plug it in, and become ~~an artist~~ a prompt copycat. With the following guide, you can, too.

How

Once a NVIDIA GPU is physically installed, set the GPU for use by the Container Station: go to Control Panel > System > Hardware > Graphics Card, under Resource Use, assign the GPU to Container Station. During the process you’ll be prompted to install 2 apps (“NVIDIA GPU Driver” and “NvKernelDriver”) from the QNAP AppCenter. These are the GPU drivers. By default, they will be installed to a .qpkg directory under the system volume. Find out the exact path with the command:

/sbin/getcfg NVIDIA_GPU_DRV Install_Path -f /etc/config/qpkg.conf -d None

In my case, the results are:

/share/CACHEDEV5_DATA/.qpkg/NVIDIA_GPU_DRV

For any Docker container to use the GPU, you need to make sure it has access to the GPU hardware, and the above location (so that it can access the drivers). To do so with a Docker Compose file, your service must include the three following changes:

Steps

Mount the devices (in the case of only 1 GPU).

devices:
  - "/dev/nvidia0:/dev/nvidia0"
  - "/dev/nvidiactl:/dev/nvidiactl"
  - "/dev/nvidia-uvm:/dev/nvidia-uvm"

Mount the driver install location to a meaningful location, such as /usr/local/nvidia. The mount can be read-only.
```
volumes:
  - "/share/CACHEDEV5_DATA/.qpkg/NVIDIA_GPU_DRV/usr:/usr/local/nvidia:ro"
```

Set the environment variable LD_LIBRARY_PATH to include the above mounted location.

environment:
  - LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64

Examples

For example, my adapted deployment of Stable Diffusion Web UI looks like this:

version: "3.9"
services:
  auto: # AUTOMATIC1111's Web UI 
    image: sd-auto:22 # this was built beforehand
    ports:
      - "7860:7860"
    volumes:
      - "/share/Container/volumes/stable-diffusion-webui-docker/data:/data"
      - "/share/Container/volumes/stable-diffusion-webui-docker/output:/output"
      - "/share/CACHEDEV5_DATA/.qpkg/NVIDIA_GPU_DRV/usr:/usr/local/nvidia:ro"
    environment:
      - CLI_ARGS=--allow-code --medvram --xformers --enable-insecure-extension-access --api
      - LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
    devices:
      - "/dev/nvidia0:/dev/nvidia0"
      - "/dev/nvidiactl:/dev/nvidiactl"
      - "/dev/nvidia-uvm:/dev/nvidia-uvm"

And (part of) my compose.yaml for Emby is just like copying the sections from above. Except the LD_LIBRARY_PATH also includes /lib and /system, because the original emby/embyserver had it set that way.

services:
  emby:
    image: emby/embyserver
    environment:      
      ...
      - LD_LIBRARY_PATH=/lib:/system:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
    volumes:
      ...
      - /share/CACHEDEV5_DATA/.qpkg/NVIDIA_GPU_DRV/usr:/usr/local/nvidia:ro
    ...
    devices:
      - "/dev/nvidia0:/dev/nvidia0"
      - "/dev/nvidiactl:/dev/nvidiactl"
      - "/dev/nvidia-uvm:/dev/nvidia-uvm"

Background

It actually took me quite some time to figure everything out. There seems to be no official documentation for this, except for a tutorial on QNAP website on running GPU-accelerated TensorFlow in Docker, which is where I got the /sbin/getcfg command from¹.

Then there was this gist on hardware transcoding in Plex and Emby, which seems to be ~~hacking~~ creatively using Docker’s volume overlay feature, so that the NVIDIA drivers installed on the host will end up at /usr in the container. However, this seems quite unnecessary because one can use the environment variable LD_LIBRARY_PATH to point to alternative locations for libraries. But it did show that you can make drivers installed on host available through volumes.

So, combining the two, now I can enjoy GPU accelerated Docker containers on a QNAP NAS, almost as if it’s a proper server. The featured image of this post is generated from Stable Diffusion running from my NAS, using the model “Anything-V3.0” and a simple prompt “alchemist”².

The tutorial may be a little outdated now if you want to follow it. To get a TensorFlow image with Jupyter as shown in the example, you now need to specify the tensorflow/tensorflow:latest-gpu-jupyter tag. ↩︎
Plus the usual stuff like “masterpiece, best quality” for positive and “nsfw, lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, bad feet” for negative prompts. ↩︎

Last modified on 2022-12-27