Deploying NVIDIA NIM Containers on DGX Spark with Minikube: A Complete Guide for AI Researchers

Minikube is a popular choice for local Kubernetes development, and multiple NVIDIA forum users have documented their setups for running NVIDIA NIM containers on DGX Spark with Minikube. This guide distills those experiences into a practical step-by-step tutorial that also shows how to deploy SLM/DSLM workloads for research and production use.

Why Minikube on DGX Spark?

Minikube gives you a full Kubernetes API on a single machine with minimal setup. On DGX Spark, it lets you test GPU-accelerated Kubernetes manifests locally before pushing them to production clusters. It is ideal for AI researchers and developers building Physical AI applications.

Step 1: Install Required Tools

# Install kubectl
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/arm64/kubectl"
sudo install kubectl /usr/local/bin/kubectl

# Install Minikube (ARM64 for DGX Spark GB10)
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-arm64
sudo install minikube-linux-arm64 /usr/local/bin/minikube

# Verify
minikube version
kubectl version --client

Step 2: Configure NVIDIA Container Runtime

sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker --set-as-default
sudo systemctl restart docker

# Test GPU in Docker
docker run --rm --gpus all nvidia/cuda:12.0-base-ubuntu22.04 nvidia-smi

Step 3: Start Minikube with GPU Support

This is the most important step. Pass the GPU parameters at Minikube startup.

minikube start \
  --driver=docker \
  --container-runtime=docker \
  --gpus all \
  --memory=64g \
  --cpus=16

# If you encounter issues, try:
minikube start --driver=none --gpus all

Step 4: Verify GPU Access Inside Minikube

minikube ssh -- nvidia-smi

# Also check Kubernetes node resources
kubectl get nodes -o json | jq '.items[].status.capacity'

Step 5: Install Helm and NVIDIA GPU Operator

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update

helm install gpu-operator nvidia/gpu-operator \
  --namespace gpu-operator \
  --create-namespace \
  --set driver.enabled=false \
  --set toolkit.enabled=false

Step 6: Create NGC API Key Secret

kubectl create secret generic ngc-api-key \
  --from-literal=NGC_API_KEY=nvapi-YOUR_KEY_HERE

kubectl create secret docker-registry ngc-registry \
  --docker-server=nvcr.io \
  --docker-username='$oauthtoken' \
  --docker-password=nvapi-YOUR_KEY_HERE

Step 7: Deploy Llama NIM via Helm

NVIDIA provides Helm charts for NIM deployments. Use them for production-ready configuration:

helm repo add nvidia-nim https://helm.ngc.nvidia.com/nim/charts
helm repo update

helm install nim-llama nvidia-nim/nvidia-nim \
  --set model.name="meta/llama-3.1-8b-instruct" \
  --set imagePullSecrets[0].name=ngc-registry \
  --set env[0].name=NGC_API_KEY \
  --set env[0].valueFrom.secretKeyRef.name=ngc-api-key \
  --set env[0].valueFrom.secretKeyRef.key=NGC_API_KEY \
  --set resources.limits."nvidia\.com/gpu"=1

Step 8: Access NIM via Port-Forward

kubectl port-forward svc/nim-llama 8000:8000 &

# Test the API
curl http://localhost:8000/v1/models

Step 9: Deploy a Research SLM Workload

For research use cases (like Physical AI in extreme environments as discussed in the NVIDIA forums), here is how to deploy a small language model optimized for edge inference:

kubectl apply -f slm-research.yaml

# slm-research.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: slm-research
  labels:
    app: slm-research
spec:
  replicas: 1
  selector:
    matchLabels:
      app: slm-research
  template:
    metadata:
      labels:
        app: slm-research
    spec:
      imagePullSecrets:
      - name: ngc-registry
      containers:
      - name: vllm
        image: nvcr.io/nvidia/vllm:25.12-py3
        args:
        - "--model"
        - "Qwen/Qwen3-4B"
        - "--max-model-len"
        - "8192"
        - "--port"
        - "8000"
        resources:
          limits:
            nvidia.com/gpu: 1
          requests:
            memory: "16Gi"
            cpu: "4"
        ports:
        - containerPort: 8000

Step 10: Add Prometheus Monitoring

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install kube-prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace \
  --set grafana.enabled=true

# Access Grafana
kubectl port-forward -n monitoring svc/kube-prometheus-grafana 3000:80 &
# Login: admin / prom-operator

Step 11: Run a Batch Inference Job

kubectl apply -f batch-job.yaml

# batch-job.yaml:
apiVersion: batch/v1
kind: Job
metadata:
  name: batch-inference
spec:
  template:
    spec:
      restartPolicy: Never
      containers:
      - name: inference
        image: python:3.11
        command:
        - python3
        - -c
        - |
          import requests
          prompts = ["Describe underwater AI systems", "Explain polar communication networks", "What is Physical AI?"]
          for p in prompts:
            r = requests.post('http://slm-research:8000/v1/completions',
              json={"model": "Qwen/Qwen3-4B", "prompt": p, "max_tokens": 150})
            print(r.json()['choices'][0]['text'])

Troubleshooting Minikube + GPU Issues

nvidia-smi not found inside minikube: Use –driver=none instead of docker. This runs Minikube directly on the host and avoids the Docker-in-Docker GPU passthrough limitation on some ARM systems.

minikube delete
minikube start --driver=none --gpus all

Helm chart pulls failing: Re-check your NGC secret. The username must be the literal string $oauthtoken (not expanded). Verify with: kubectl get secret ngc-registry -o yaml

Pod in Pending state: Check if Minikube sees the GPU resource: kubectl describe node minikube | grep nvidia

Minikube vs K3s on DGX Spark: When to Use What

Conclusion

Minikube on NVIDIA DGX Spark gives researchers and developers a full Kubernetes environment with GPU acceleration in under 30 minutes. The combination of DGX Spark unified memory architecture and NVIDIA NIM containers makes it an ideal platform for developing Physical AI applications, testing distributed inference patterns, and building production-ready AI microservices.