GPU Sharing in Kubernetes: The Complete Hands-On Tutorial (Time-Slicing, MIG, HAMi & DRA)

GPUs are the most expensive line item in most AI infrastructure budgets, yet a single pod rarely saturates a modern GPU like an A100 or H100 for inference or light training workloads. Kubernetes, by default, treats a GPU as a single indivisible resource. This guide walks through the four approaches the CNCF community is actively discussing right now for sharing GPUs across pods: time-slicing, MIG, HAMi, and Dynamic Resource Allocation (DRA).

Part 1: GPU Time-Slicing with the NVIDIA GPU Operator

Time-slicing lets the NVIDIA GPU Operator present one physical GPU as several schedulable replicas, so multiple pods share it concurrently via fast context switching at the driver level. It works on almost any NVIDIA GPU and requires no special hardware.

Step 1: Install the GPU Operator

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update
kubectl create namespace gpu-operator
helm install gpu-operator nvidia/gpu-operator \
  -n gpu-operator \
  --set driver.enabled=true \
  --set toolkit.enabled=true

Step 2: Configure time-slicing

apiVersion: v1
kind: ConfigMap
metadata:
  name: time-slicing-config
  namespace: gpu-operator
data:
  a100: |-
    version: v1
    flags:
      migStrategy: none
    sharing:
      timeSlicing:
        resources:
        - name: nvidia.com/gpu
          replicas: 4

kubectl apply -f time-slicing-config.yaml
kubectl patch clusterpolicy/cluster-policy -n gpu-operator --type merge \
  -p '{"spec": {"devicePlugin": {"config": {"name": "time-slicing-config", "default": "a100"}}}}'

Step 3: Verify and deploy

kubectl get node NODE_NAME -o json | jq '.status.capacity'

Deploy several pods each requesting nvidia.com/gpu: 1 and confirm with kubectl exec -it POD -- nvidia-smi that they report the same GPU UUID — proof they are sharing the same physical device.

Time-slicing is best for bursty, latency-tolerant inference, CI, and notebooks. It offers no memory or fault isolation between pods.

Part 2: NVIDIA MIG — Hardware-Level GPU Partitioning

Multi-Instance GPU (MIG) physically partitions a single Ampere/Hopper/Blackwell-class GPU (A100, H100, H200, and newer) into up to seven isolated instances, each with dedicated memory and compute cores — ideal for hard multi-tenancy.

Step 1: Choose a strategy and enable MIG support

helm upgrade gpu-operator nvidia/gpu-operator -n gpu-operator --set mig.strategy=mixed

Step 2: Label the node with a MIG profile

kubectl label node NODE_NAME nvidia.com/mig.config=all-1g.5gb --overwrite

The MIG Manager daemonset detects the label, cordons the node, applies the partition, and uncordons it automatically.

Step 3: Request a MIG slice

apiVersion: v1
kind: Pod
metadata:
  name: mig-inference-test
spec:
  containers:
  - name: cuda-test
    image: nvidia/cuda:12.4.0-base-ubuntu22.04
    command: ["sleep", "3600"]
    resources:
      limits:
        nvidia.com/mig-1g.5gb: 1

Validate isolation with kubectl exec -it mig-inference-test -- nvidia-smi -L — each slice reports its own distinct device UUID.

MIG vs. time-slicing vs. MPS: MIG gives hardware-level isolation but only on select GPU families with fixed partition shapes. Time-slicing works broadly but has zero isolation. NVIDIA MPS sits in between, offering concurrent execution with a shared fault domain.

Part 3: HAMi — Fine-Grained GPU Sharing (CNCF Sandbox Project)

HAMi (formerly k8s-vGPU-scheduler) is a CNCF sandbox project solving fractional GPU allocation by memory and compute percentage, with enforced isolation via its HAMi-core runtime layer.

Step 1: Install HAMi

helm repo add hami-charts https://project-hami.github.io/HAMi/
helm repo update
kubectl create namespace hami-system
helm install hami hami-charts/hami -n hami-system

Step 2: Label GPU nodes

kubectl label node NODE_NAME gpu=on --overwrite

Step 3: Request a fractional GPU

apiVersion: v1
kind: Pod
metadata:
  name: hami-fractional-test
spec:
  schedulerName: hami-scheduler
  containers:
  - name: cuda-test
    image: nvidia/cuda:12.4.0-base-ubuntu22.04
    command: ["sleep", "3600"]
    resources:
      limits:
        nvidia.com/gpu: 1
        nvidia.com/gpumem: 4000
        nvidia.com/gpucores: 30

This pod is guaranteed 4GB of GPU memory and capped at roughly 30% compute throughput, enforced at runtime by HAMi-core — not just a scheduling suggestion.

Check live allocation across tenants with the HAMi WebUI: kubectl port-forward -n hami-system svc/hami-webui 8080:8080.

Part 4: Dynamic Resource Allocation (DRA) — the New GA Standard

Dynamic Resource Allocation reached General Availability in Kubernetes 1.35, and NVIDIA has donated its DRA driver to the CNCF, making DRA the center of GPU scheduling conversations across the community. Instead of requesting a fixed nvidia.com/gpu: 1, DRA lets you describe exactly what you need — a specific GPU model, MIG profile, or even multiple NVLink-connected GPUs — and the scheduler negotiates the allocation dynamically.

Step 1: Install the NVIDIA DRA driver

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update
helm install nvidia-dra-driver nvidia/nvidia-dra-driver-gpu -n nvidia-dra-driver --create-namespace

Step 2: Create a ResourceClaimTemplate

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: single-gpu-claim
spec:
  spec:
    devices:
      requests:
      - name: gpu-request
        deviceClassName: gpu.nvidia.com

Step 3: Reference the claim from a pod

apiVersion: v1
kind: Pod
metadata:
  name: dra-gpu-test
spec:
  containers:
  - name: cuda-test
    image: nvidia/cuda:12.4.0-base-ubuntu22.04
    command: ["sleep", "3600"]
    resources:
      claims:
      - name: gpu-request
  resourceClaims:
  - name: gpu-request
    resourceClaimTemplateName: single-gpu-claim

Inspect fulfillment with kubectl describe resourceclaim CLAIM_NAME — the status shows exactly which physical GPU was allocated and by which driver, giving full auditability that the old device-plugin API never exposed.

DRA can even express topology-aware requests like two NVLink-connected GPUs, something impossible under the legacy model. Pair it with the KAI Scheduler for batch/gang scheduling and fair-share queuing across teams.

Which Approach Should You Use?

Choose time-slicing for maximizing utilization on best-effort, latency-tolerant workloads. Choose MIG when you need hardware-guaranteed isolation on supported GPUs. Choose HAMi when you want fine-grained fractional sharing with enforced limits across any NVIDIA GPU. Choose DRA when you’re building on the new Kubernetes-native resource model that the whole ecosystem, including HAMi and KAI Scheduler, is converging toward.