NVIDIA DGX Spark + Kubernetes: Run GPU Workloads on the GB10 Grace Blackwell Superchip

The NVIDIA DGX Spark puts a 1 PFLOP (FP4) Grace Blackwell GB10 superchip and 128 GB of unified memory on your desk, enough to run models up to 200B parameters locally. But the moment you try to schedule that GPU through Kubernetes, you hit a wall that doesn’t exist on a normal discrete GPU: the device plugin crashes with error getting device memory: Not Supported.

This guide fixes that and gets you to a working GPU cluster, fast. More code, less talk.

TL;DR — DGX Spark’s GB10 uses a Unified Memory Architecture (UMA). The classic NVIDIA device plugin calls nvmlDeviceGetMemoryInfo, which returns Not Supported on UMA and kills the plugin. The fix: use k8s-device-plugin ≥ v0.17.4 (or a GPU Operator release that bundles it). Everything below uses K3s + the GPU Operator on arm64 DGX OS.

Hardware & software baseline

Step 0 — Confirm the GPU works on the host

# Driver + GPU present?
nvidia-smi

Expected output on DGX Spark — note Memory-Usage: Not Supported. This is normal for a unified-memory iGPU (no dedicated framebuffer), not a bug:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 12.4       |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name              Persistence-M    | Bus-Id          Disp.A | Volatile Uncorr. ECC |
|=========================================+========================+======================|
|   0  NVIDIA GB10              On         | 0000000F:01:00.0  Off  |                  N/A |
| N/A  40C    P8     4W /  N/A             |     Not Supported      |   0%      Default    |
+-----------------------------------------+------------------------+----------------------+

# Confirm the container toolkit is installed (ships with DGX OS)
nvidia-ctk --version
dpkg -l | grep -i nvidia-container-toolkit

Step 1 — Point containerd at the NVIDIA runtime

K3s ships its own embedded containerd. Register the NVIDIA runtime so it survives K3s restarts:

# Generate NVIDIA runtime config for K3s' containerd template
sudo nvidia-ctk runtime configure \
  --runtime=containerd \
  --config=/var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl \
  --set-as-default

If you’d rather use Docker as the runtime (validated path on the NVIDIA forums), set this in /etc/docker/daemon.json instead:

{
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "args": []
    }
  },
  "default-runtime": "nvidia"
}

sudo systemctl restart docker

Step 2 — Install K3s

containerd path (recommended, Kubernetes-native):

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--write-kubeconfig-mode 644 --disable=traefik" sh -

# Wait for the node to go Ready
sudo k3s kubectl get nodes -o wide

Docker path (alternative):

curl -sfL https://get.k3s.io | \
  INSTALL_K3S_EXEC="--docker --write-kubeconfig-mode 644 --disable=traefik" sh -

Wire up kubectl:

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
kubectl get nodes
# NAME    STATUS   ROLES                  AGE   VERSION
# spark   Ready    control-plane,master   1m    v1.30.x+k3s1

Step 3 — The unified-memory gotcha (and the fix)

Here’s the failure everyone hits. The stock device plugin enumerates devices via nvmlDeviceGetMemoryInfo, which returns Not Supported on GB10’s shared 128 GB pool:

E1029 10:26:46.968061  1 main.go:173] error starting plugins: ...
  error building GPU device map: error visiting device:
  error getting info for GPU 0: error getting memory info for device 0: Not Supported

Root cause: GB10 shares DRAM between CPU and GPU. There is no discrete VRAM to report, so the NVML memory call fails.

Fix: This was resolved in k8s-device-plugin v0.17.4 — newer plugins gracefully handle the Not Supported response and still register the GPU. Pin to a GPU Operator release that bundles v0.17.4+, or run the standalone plugin at that version.

# Verify which device-plugin version you're about to deploy
helm repo add nvidia https://helm.ngc.nvidia.com/nvidia && helm repo update
helm search repo nvidia/gpu-operator --versions | head

⚠️ The DRA driver (dra-driver-nvidia-gpu) hit the same UMA issue separately — if you go the DRA route instead of the classic plugin, confirm your version includes the GB10 fix before deploying.

Step 4 — Install the GPU Operator (driver + toolkit pre-installed)

DGX OS already has the driver and container toolkit baked in, so disable both in the Operator and let it manage only the plugin + validators:

kubectl create ns gpu-operator
kubectl label --overwrite ns gpu-operator \
  pod-security.kubernetes.io/enforce=privileged

helm install --wait gpu-operator \
  -n gpu-operator \
  nvidia/gpu-operator \
  --set driver.enabled=false \
  --set toolkit.enabled=false \
  --set devicePlugin.version=v0.17.4

Watch the pods come up:

kubectl get pods -n gpu-operator -w

Confirm the node now advertises GPU capacity:

kubectl get node spark -o jsonpath='{.status.capacity.nvidia\.com/gpu}{"\n"}'
# 1

If it prints 1, the unified-memory problem is solved. 🎉

Step 5 — Smoke test: schedule a GPU pod

# gpu-smoke.yaml
apiVersion: v1
kind: Pod
metadata:
  name: cuda-smoke
spec:
  restartPolicy: Never
  containers:
    - name: cuda
      image: nvcr.io/nvidia/cuda:12.4.1-base-ubuntu22.04
      command: ["nvidia-smi"]
      resources:
        limits:
          nvidia.com/gpu: 1

kubectl apply -f gpu-smoke.yaml
kubectl logs cuda-smoke
# You should see the GB10 listed inside the pod.

Step 6 — Share one GPU across many pods (time-slicing)

DGX Spark has a single GPU, so to run multiple pods you enable time-slicing. Create the config:

# time-slicing-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: time-slicing-config
  namespace: gpu-operator
data:
  any: |-
    version: v1
    flags:
      migStrategy: none
    sharing:
      timeSlicing:
        resources:
          - name: nvidia.com/gpu
            replicas: 4   # expose the GB10 as 4 schedulable slices

kubectl apply -f time-slicing-config.yaml

# Point the Operator's device plugin at the config
kubectl patch clusterpolicy/cluster-policy \
  -n gpu-operator --type merge \
  -p '{"spec":{"devicePlugin":{"config":{"name":"time-slicing-config","default":"any"}}}}'

Verify the GPU now advertises 4 replicas:

kubectl get node spark -o jsonpath='{.status.capacity.nvidia\.com/gpu}{"\n"}'
# 4

Time-slicing is temporal sharing — no hard memory isolation between slices. On 128 GB of unified memory that’s usually fine for dev/inference, but size your pods so they don’t collectively OOM the box.

Step 7 — Deploy a real workload: vLLM serving Qwen3

Now serve an actual LLM on the cluster. This pulls a model from Hugging Face and exposes an OpenAI-compatible API.

# vllm-qwen3.yaml
apiVersion: v1
kind: Secret
metadata:
  name: hf-token
stringData:
  token: "hf_xxxxxxxxxxxxxxxxxxxxxxxx"   # your Hugging Face token
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: qwen3-4b
  labels: { app: qwen3-4b }
spec:
  replicas: 1
  selector: { matchLabels: { app: qwen3-4b } }
  template:
    metadata:
      labels: { app: qwen3-4b }
    spec:
      containers:
        - name: vllm
          image: vllm/vllm-openai:latest
          args:
            - "--model=Qwen/Qwen3-4B"
            - "--gpu-memory-utilization=0.85"
            - "--max-model-len=8192"
          ports:
            - containerPort: 8000
          env:
            - name: HUGGING_FACE_HUB_TOKEN
              valueFrom:
                secretKeyRef: { name: hf-token, key: token }
          resources:
            limits:
              cpu: "8"
              memory: 16Gi
              nvidia.com/gpu: 1
            requests:
              cpu: "4"
              memory: 8Gi
              nvidia.com/gpu: 1
          volumeMounts:
            - { name: cache, mountPath: /root/.cache/huggingface }
            - { name: shm,   mountPath: /dev/shm }
      volumes:
        - name: cache
          emptyDir: {}
        - name: shm
          emptyDir: { medium: Memory, sizeLimit: 8Gi }
---
apiVersion: v1
kind: Service
metadata:
  name: qwen3-4b
spec:
  type: ClusterIP
  selector: { app: qwen3-4b }
  ports:
    - { name: http, port: 8000, targetPort: 8000 }

kubectl apply -f vllm-qwen3.yaml
kubectl get pods -w
# qwen3-4b-7fd7d4485d-j42fn   1/1   Running   0   2m

Hit the API:

kubectl port-forward svc/qwen3-4b 8000:8000 &

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-4B",
    "messages": [{"role": "user", "content": "Explain DGX Spark in one line."}]
  }'

Step 8 — GPU observability with DCGM exporter

DGX OS already runs nv-hostengine, so point the DCGM exporter at the host engine instead of starting its own:

helm repo add gpu-helm-charts https://nvidia.github.io/dcgm-exporter/helm-charts
helm repo update

helm install dcgm-exporter gpu-helm-charts/dcgm-exporter \
  -n gpu-operator

# Scrape metrics
kubectl port-forward -n gpu-operator svc/dcgm-exporter 9400:9400 &
curl -s http://localhost:9400/metrics | grep DCGM_FI_DEV_GPU_UTIL

For live terminal monitoring on the host (handles UMA correctly when built fresh):

git clone https://github.com/Syllo/nvtop.git && cd nvtop
sudo apt-get update && sudo apt-get install -y libncurses-dev libdrm-dev cmake
mkdir build && cd build
cmake .. -DNVIDIA_SUPPORT=ON -DAMDGPU_SUPPORT=OFF -DINTEL_SUPPORT=OFF
make -j && ./src/nvtop

Troubleshooting cheat sheet

FAQ

Can you run full Kubernetes (kubeadm) on DGX Spark instead of K3s? Yes, but K3s is the path of least resistance on a single arm64 node — lower overhead, embedded containerd, and a one-line install. The GPU Operator works identically on both.

Why does nvidia-smi say memory is “Not Supported”? GB10 is a unified-memory iGPU with no dedicated framebuffer. NVML’s per-device memory query has nothing discrete to report. Per-process GPU memory still shows up; the aggregate field doesn’t.

Can two DGX Sparks form one cluster? Yes — the built-in ConnectX-7 200GbE links two Sparks, letting you run models up to ~405B parameters and join both as Kubernetes nodes.

Is MIG (Multi-Instance GPU) available? No. GB10 doesn’t expose MIG, so time-slicing is your GPU-sharing mechanism on a single Spark.

Does the GPU Operator need to install drivers? No — DGX OS ships the driver and container toolkit. Install with driver.enabled=false and toolkit.enabled=false.

Wrapping up

DGX Spark is a legitimate Kubernetes GPU node once you clear the one UMA-shaped speed bump. The whole recipe:

# 1. NVIDIA runtime for K3s' containerd
sudo nvidia-ctk runtime configure --runtime=containerd \
  --config=/var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl --set-as-default
# 2. K3s
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--write-kubeconfig-mode 644 --disable=traefik" sh -
# 3. GPU Operator with the FIXED device plugin
helm install --wait gpu-operator -n gpu-operator --create-namespace nvidia/gpu-operator \
  --set driver.enabled=false --set toolkit.enabled=false --set devicePlugin.version=v0.17.4
# 4. Schedule GPUs. Done.

From here, layer on time-slicing, vLLM, DCGM dashboards and you’ve got a data-center-grade AI stack running on a 150 mm box on your desk.

Found this useful? Share it, and drop your DGX Spark + Kubernetes war stories in the comments.

Related Posts

If you found this guide helpful, explore these related articles on Collabnix:

• Building AI Agents on DGX Spark with Kubernetes: A Complete Tutorial — Learn how to deploy fully local agentic AI workflows on the same GB10 Grace Blackwell platform.
• Getting Started with NVIDIA DGX Spark: Your Personal AI Supercomputer — A complete unboxing-to-running guide for first-time DGX Spark users.
• How to Run GPU Workloads on Kubernetes with NVIDIA: A Step-by-Step Guide — A broader look at scheduling GPU workloads on Kubernetes using NVIDIA hardware.
• Kubernetes for Generative AI: Complete Guide to Deploying LLMs at Scale — Understand how Kubernetes handles LLM workloads at production scale.