Running vLLM on NVIDIA DGX Spark with Docker: Zero-Compilation Guide for CUDA 13

Table of Contents

Users on the NVIDIA Developer Forums regularly report that getting pre-built vLLM Docker images running on DGX Spark was tricky in the early days due to CUDA 13 compatibility issues. This guide shows you exactly how to run the official NVIDIA vLLM Docker images on DGX Spark with zero compilation required.

Why vLLM on DGX Spark?

The DGX Spark ships with the GB10 Grace Blackwell Superchip running CUDA 13. Early vLLM builds were not pre-compiled for CUDA 13, forcing users to compile from source. NVIDIA now ships pre-built vLLM images on NGC that work out of the box. Here is how to use them.

Step 1: Check Your CUDA Version

nvcc --version
nvidia-smi
# Confirm CUDA 13.x on DGX Spark GB10

Step 2: Log in to NGC Registry

docker login nvcr.io
# Username: $oauthtoken
# Password: YOUR_NGC_API_KEY

Step 3: Pull the Official NVIDIA vLLM Image

Use the latest tag that supports your CUDA version. For DGX Spark, use the 25.12-py3 or newer tag.

docker pull nvcr.io/nvidia/vllm:25.12-py3

# Verify the image
docker images | grep vllm

Step 4: Run Llama 3.1 8B with vLLM

docker run -d \
  --gpus all \
  --name vllm-server \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN=YOUR_HF_TOKEN \
  nvcr.io/nvidia/vllm:25.12-py3 \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --port 8000 \
  --max-model-len 8192

Step 5: Check Startup Logs

docker logs -f vllm-server
# Wait for: "INFO: Application startup complete"
# This can take 3-5 minutes for model loading

Step 6: Test the API

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "meta-llama/Llama-3.1-8B-Instruct", "messages": [{"role": "user", "content": "What makes DGX Spark special for AI workloads?"}], "max_tokens": 200}'

Step 7: Run DeepSeek-R1 Instead

For DeepSeek R1 (a popular choice on DGX Spark), adjust the model path:

docker run -d \
  --gpus all \
  --name vllm-deepseek \
  -p 8001:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN=YOUR_HF_TOKEN \
  nvcr.io/nvidia/vllm:25.12-py3 \
  --model deepseek-ai/DeepSeek-R1-Distill-Llama-8B \
  --port 8000 \
  --max-model-len 16384

Step 8: Run Multiple Models Simultaneously

DGX Spark has 128GB unified memory — plenty for multiple small models.

docker run -d --gpus all --name vllm-qwen -p 8002:8000 \
  -e HF_TOKEN=YOUR_HF_TOKEN \
  nvcr.io/nvidia/vllm:25.12-py3 \
  --model Qwen/Qwen3-4B --port 8000

# Check GPU memory usage
nvidia-smi --query-gpu=memory.used,memory.free --format=csv

Step 9: Use Docker Compose for Multiple Models

cat docker-compose.yml

version: "3.8"
services:
  llama:
    image: nvcr.io/nvidia/vllm:25.12-py3
    runtime: nvidia
    ports: ["8000:8000"]
    command: ["--model", "meta-llama/Llama-3.1-8B-Instruct", "--port", "8000"]
    environment:
      - HF_TOKEN=YOUR_HF_TOKEN
  qwen:
    image: nvcr.io/nvidia/vllm:25.12-py3
    runtime: nvidia
    ports: ["8001:8000"]
    command: ["--model", "Qwen/Qwen3-4B", "--port", "8000"]
    environment:
      - HF_TOKEN=YOUR_HF_TOKEN

docker compose up -d

Step 10: Monitor GPU and Container Health

# Watch GPU utilization
watch -n 1 nvidia-smi

# Check container stats
docker stats

# Run a quick benchmark
python3 -c "
import requests, time
start = time.time()
for i in range(10):
    r = requests.post('http://localhost:8000/v1/completions',
        json={'model': 'meta-llama/Llama-3.1-8B-Instruct', 'prompt': 'Hello', 'max_tokens': 50})
print(f'10 requests in {time.time()-start:.2f}s')
"

Common Issues and Fixes

CUDA version mismatch: Always use the image tag that matches your CUDA version. Check with nvcc –version and pick the corresponding NGC tag.

Model download slow: Models are downloaded from HuggingFace on first run. Pre-download with: huggingface-cli download meta-llama/Llama-3.1-8B-Instruct

Port already in use: Check with: ss -tlnp | grep 8000 and kill the process or change the host port mapping.

Conclusion

With NVIDIA’s official vLLM Docker images on NGC, you get zero-compilation LLM serving on DGX Spark. The pre-built CUDA 13 images eliminate the biggest pain point the community experienced early on. You can serve multiple large language models simultaneously thanks to DGX Spark’s 128GB unified memory architecture.