Testcontainers Tutorial: Docker Model Runner Guide

Join our Meetup Group

Docker Model Runner is an experimental feature introduced in Docker Desktop 4.40+ that provides a Docker-native experience for running Large Language Models (LLMs) locally. Unlike traditional containerized approaches, Docker Model Runner runs AI models as host-level processes using optimized inference engines (currently llama.cpp), providing direct GPU acceleration on Apple Silicon Macs.

With the Model Runner feature, Docker provides inference capabilities to developers on their laptop, and in the future in CI, allowing them to run LLM models locally. This is an important feature to help developing GenAI applications. The runner essentially provides GPU-accelerated inference engines that are accessible both through the Docker socket (/var/run/docker.sock) and via a TCP connection at model-runner.docker.internal:80.

What is Docker Model Runner and what problem does it solve?

Key Architecture Points:

Models don’t run in containers – they run directly on the host for optimal performance
GPU-accelerated inference – Direct access to Apple’s Metal API on M-series chips
Host-installed inference server – Uses llama.cpp natively on your Mac
OCI artifact storage – Models are stored as standardized artifacts in Docker Hub

This tutorial focuses on the Testcontainers module, which allows you to programmatically manage Docker Model Runner instances in your Go applications.

Comprehensive Testcontainers Tutorial for Docker Users

Prerequisites

Before starting this tutorial, ensure you have:

Go 1.19+ installed on your system
Docker installed and running
Basic Go knowledge (modules, contexts, error handling)
Internet connection for pulling model artifacts

Step 1: Project Setup

1.1 Initialize a New Go Module

mkdir dockermodelrunner-tutorial
cd dockermodelrunner-tutorial
go mod init dockermodelrunner-tutorial
....
go: creating new go.mod: module dockermodelrunner-tutorial

1.2 Add the Required Dependencies

go get github.com/testcontainers/testcontainers-go/modules/dockermodelrunner...


go: downloading github.com/yusufpapurcu/wmi v1.2.4
go: added dario.cat/mergo v1.0.1
go: added github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161
go: added github.com/Microsoft/go-winio v0.6.2
go: added github.com/cenkalti/backoff/v4 v4.2.1
go: added github.com/containerd/log v0.1.0
go: added github.com/containerd/platforms v0.2.1
go: added github.com/cpuguy83/dockercfg v0.3.2
go: added github.com/davecgh/go-spew v1.1.1
go: added github.com/distribution/reference v0.6.0
go: added github.com/docker/docker v28.0.1+incompatible
go: added github.com/docker/go-connections v0.5.0
go: added github.com/docker/go-units v0.5.0
go: added github.com/ebitengine/purego v0.8.2
go: added github.com/felixge/httpsnoop v1.0.4
go: added github.com/go-logr/logr v1.4.2
go: added github.com/go-logr/stdr v1.2.2
go: added github.com/go-ole/go-ole v1.2.6
go: added github.com/gogo/protobuf v1.3.2
go: added github.com/google/uuid v1.6.0
go: added github.com/klauspost/compress v1.17.6
go: added github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0
go: added github.com/magiconair/properties v1.8.10
go: added github.com/moby/docker-image-spec v1.3.1
go: added github.com/moby/patternmatcher v0.6.0
go: added github.com/moby/sys/sequential v0.5.0
go: added github.com/moby/sys/user v0.1.0
go: added github.com/moby/sys/userns v0.1.0
go: added github.com/moby/term v0.5.0
go: added github.com/morikuni/aec v1.0.0
go: added github.com/opencontainers/go-digest v1.0.0
go: added github.com/opencontainers/image-spec v1.1.1
go: added github.com/pkg/errors v0.9.1
go: added github.com/pmezard/go-difflib v1.0.0
go: added github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c
go: added github.com/shirou/gopsutil/v4 v4.25.1
go: added github.com/sirupsen/logrus v1.9.3
go: added github.com/stretchr/testify v1.10.0
go: added github.com/testcontainers/testcontainers-go v0.37.0
go: added github.com/testcontainers/testcontainers-go/modules/dockermodelrunner v0.37.2
go: added github.com/testcontainers/testcontainers-go/modules/socat v0.37.0
go: added github.com/tklauser/go-sysconf v0.3.12
go: added github.com/tklauser/numcpus v0.6.1
go: added github.com/yusufpapurcu/wmi v1.2.4
go: added go.opentelemetry.io/auto/sdk v1.1.0
go: added go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.51.0
go: added go.opentelemetry.io/otel v1.35.0
go: added go.opentelemetry.io/otel/metric v1.35.0
go: added go.opentelemetry.io/otel/trace v1.35.0
go: added golang.org/x/crypto v0.37.0
go: added golang.org/x/sys v0.32.0
go: added gopkg.in/yaml.v3 v3.0.1

Step 2: Basic Container Setup

2.1 Create Your First Docker Model Runner Container

Create a file named main.go:

package main

import (
    "context"
    "log"
    "time"

    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/modules/dockermodelrunner"
)

func main() {
    ctx := context.Background()

    // Define model details
    const (
        modelNamespace = "ai"
        modelName      = "smollm2"
        modelTag       = "360M-Q4_K_M"
        fqModelName    = modelNamespace + "/" + modelName + ":" + modelTag
    )

    // Create and start the Docker Model Runner container
    dmrCtr, err := dockermodelrunner.Run(
        ctx,
        dockermodelrunner.WithModel(fqModelName),
    )
    if err != nil {
        log.Fatalf("failed to start container: %s", err)
    }

    // Ensure proper cleanup
    defer func() {
        if err := testcontainers.TerminateContainer(dmrCtr); err != nil {
            log.Printf("failed to terminate container: %s", err)
        }
    }()

    log.Println("Docker Model Runner container started successfully!")
    
    // Keep the container running for demonstration
    time.Sleep(10 * time.Second)
}

2.2 Run Your First Example

go run main.go

Step 3: Working with Models at Runtime

3.1 Pulling Models Dynamically

Create model_operations.go:

package main

import (
    "context"
    "log"
    "time"

    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/modules/dockermodelrunner"
)

func pullModelExample() {
    ctx := context.Background()

    // Start container without pre-loading a model
    dmrCtr, err := dockermodelrunner.Run(ctx)
    if err != nil {
        log.Fatalf("failed to start container: %s", err)
    }
    defer func() {
        if err := testcontainers.TerminateContainer(dmrCtr); err != nil {
            log.Printf("failed to terminate container: %s", err)
        }
    }()

    // Define models to pull
    models := []string{
        "ai/smollm2:360M-Q4_K_M",
        "ai/llama3.2:latest",
    }

    for _, modelName := range models {
        log.Printf("Pulling model: %s", modelName)
        
        // Create a timeout context for the pull operation
        pullCtx, cancel := context.WithTimeout(ctx, 60*time.Second)
        
        err = dmrCtr.PullModel(pullCtx, modelName)
        cancel() // Always cancel to free resources
        
        if err != nil {
            log.Printf("failed to pull model %s: %s", modelName, err)
            continue
        }
        
        log.Printf("Successfully pulled model: %s", modelName)
    }
}

3.2 Inspecting Models

Add this function to your file:

func inspectModelExample() {
    ctx := context.Background()

    dmrCtr, err := dockermodelrunner.Run(ctx)
    if err != nil {
        log.Fatalf("failed to start container: %s", err)
    }
    defer func() {
        if err := testcontainers.TerminateContainer(dmrCtr); err != nil {
            log.Printf("failed to terminate container: %s", err)
        }
    }()

    // Pull a model first
    modelNamespace := "ai"
    modelName := "smollm2"
    modelTag := "360M-Q4_K_M"
    fqModelName := modelNamespace + "/" + modelName + ":" + modelTag

    err = dmrCtr.PullModel(ctx, fqModelName)
    if err != nil {
        log.Printf("failed to pull model: %s", err)
        return
    }

    // Inspect the model
    model, err := dmrCtr.InspectModel(ctx, modelNamespace, modelName+":"+modelTag)
    if err != nil {
        log.Printf("failed to inspect model: %s", err)
        return
    }

    log.Printf("Model inspection result: %+v", model)
}

3.3 Listing Available Models

Add this function:

func listModelsExample() {
    ctx := context.Background()

    dmrCtr, err := dockermodelrunner.Run(ctx)
    if err != nil {
        log.Fatalf("failed to start container: %s", err)
    }
    defer func() {
        if err := testcontainers.TerminateContainer(dmrCtr); err != nil {
            log.Printf("failed to terminate container: %s", err)
        }
    }()

    // Pull some models
    models := []string{
        "ai/smollm2:360M-Q4_K_M",
        "ai/llama3.2:latest",
    }

    for _, modelName := range models {
        err = dmrCtr.PullModel(ctx, modelName)
        if err != nil {
            log.Printf("failed to pull model %s: %s", modelName, err)
        }
    }

    // List all locally available models
    localModels, err := dmrCtr.ListModels(ctx)
    if err != nil {
        log.Printf("failed to list models: %s", err)
        return
    }

    log.Println("Locally available models:")
    for i, model := range localModels {
        log.Printf("  %d. %+v", i+1, model)
    }
}

Step 4: Advanced Configuration

4.1 Container Customization

Create advanced_config.go:

package main

import (
    "context"
    "log"
    "time"

    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/modules/dockermodelrunner"
    "github.com/testcontainers/testcontainers-go/wait"
)

func advancedConfigExample() {
    ctx := context.Background()

    dmrCtr, err := dockermodelrunner.Run(
        ctx,
        dockermodelrunner.WithModel("ai/smollm2:360M-Q4_K_M"),
        testcontainers.WithEnv(map[string]string{
            "CUSTOM_ENV_VAR": "custom_value",
        }),
        testcontainers.WithExposedPorts("8080/tcp"),
        testcontainers.WithWaitStrategy(
            wait.ForLog("Model loaded successfully").WithStartupTimeout(60*time.Second),
        ),
        testcontainers.WithLabels(map[string]string{
            "app":     "dockermodelrunner",
            "version": "tutorial",
        }),
    )
    if err != nil {
        log.Fatalf("failed to start container: %s", err)
    }
    defer func() {
        if err := testcontainers.TerminateContainer(dmrCtr); err != nil {
            log.Printf("failed to terminate container: %s", err)
        }
    }()

    log.Println("Advanced configured container started successfully!")
}

4.2 Error Handling and Timeouts

Create error_handling.go:

package main

import (
    "context"
    "errors"
    "log"
    "time"

    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/modules/dockermodelrunner"
)

func robustModelOperations() {
    ctx := context.Background()

    // Start container with timeout
    startCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
    defer cancel()

    dmrCtr, err := dockermodelrunner.Run(startCtx)
    if err != nil {
        log.Fatalf("failed to start container: %s", err)
    }
    defer func() {
        if err := testcontainers.TerminateContainer(dmrCtr); err != nil {
            log.Printf("failed to terminate container: %s", err)
        }
    }()

    // Robust model pulling with retries
    modelName := "ai/smollm2:360M-Q4_K_M"
    maxRetries := 3
    retryDelay := 5 * time.Second

    for attempt := 1; attempt <= maxRetries; attempt++ {
        log.Printf("Attempt %d to pull model: %s", attempt, modelName)
        
        pullCtx, cancel := context.WithTimeout(ctx, 120*time.Second)
        err = dmrCtr.PullModel(pullCtx, modelName)
        cancel()
        
        if err == nil {
            log.Printf("Successfully pulled model on attempt %d", attempt)
            break
        }
        
        log.Printf("Attempt %d failed: %s", attempt, err)
        
        if attempt < maxRetries {
            log.Printf("Waiting %v before retry...", retryDelay)
            time.Sleep(retryDelay)
        } else {
            log.Fatalf("Failed to pull model after %d attempts", maxRetries)
        }
    }

    // Verify model was pulled
    models, err := dmrCtr.ListModels(ctx)
    if err != nil {
        log.Printf("failed to list models: %s", err)
        return
    }

    if len(models) == 0 {
        log.Println("No models found locally")
        return
    }

    log.Printf("Found %d models locally", len(models))
}

Step 5: Complete Working Example

Create complete_example.go:

package main

import (
    "context"
    "fmt"
    "log"
    "time"

    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/modules/dockermodelrunner"
)

func completeWorkflow() {
    ctx := context.Background()

    log.Println("🚀 Starting Docker Model Runner Tutorial")

    // Step 1: Create container
    log.Println("📦 Creating Docker Model Runner container...")
    dmrCtr, err := dockermodelrunner.Run(ctx)
    if err != nil {
        log.Fatalf("❌ Failed to start container: %s", err)
    }
    defer func() {
        log.Println("🧹 Cleaning up container...")
        if err := testcontainers.TerminateContainer(dmrCtr); err != nil {
            log.Printf("⚠️  Failed to terminate container: %s", err)
        } else {
            log.Println("✅ Container terminated successfully")
        }
    }()

    log.Println("✅ Container started successfully")

    // Step 2: Pull multiple models
    models := []string{
        "ai/smollm2:360M-Q4_K_M",
    }

    for _, modelName := range models {
        log.Printf("⬇️  Pulling model: %s", modelName)
        
        pullCtx, cancel := context.WithTimeout(ctx, 180*time.Second)
        err = dmrCtr.PullModel(pullCtx, modelName)
        cancel()
        
        if err != nil {
            log.Printf("❌ Failed to pull model %s: %s", modelName, err)
            continue
        }
        log.Printf("✅ Successfully pulled: %s", modelName)
    }

    // Step 3: List all models
    log.Println("📋 Listing all available models...")
    localModels, err := dmrCtr.ListModels(ctx)
    if err != nil {
        log.Printf("❌ Failed to list models: %s", err)
        return
    }

    fmt.Printf("📊 Found %d models:\n", len(localModels))
    for i, model := range localModels {
        fmt.Printf("   %d. %+v\n", i+1, model)
    }

    // Step 4: Inspect a specific model
    if len(localModels) > 0 {
        // For inspection, we need namespace and name:tag format
        modelNamespace := "ai"
        modelNameTag := "smollm2:360M-Q4_K_M"
        
        log.Printf("🔍 Inspecting model: %s/%s", modelNamespace, modelNameTag)
        model, err := dmrCtr.InspectModel(ctx, modelNamespace, modelNameTag)
        if err != nil {
            log.Printf("❌ Failed to inspect model: %s", err)
        } else {
            log.Printf("✅ Model details: %+v", model)
        }
    }

    log.Println("🎉 Tutorial completed successfully!")
}

func main() {
    completeWorkflow()
}

Step 6: Running the Tutorial

6.1 Run the Complete Example

go run complete_example.go

6.2 Expected Output

You should see output similar to:

🚀 Starting Docker Model Runner Tutorial
📦 Creating Docker Model Runner container...
✅ Container started successfully
⬇️  Pulling model: ai/smollm2:360M-Q4_K_M
✅ Successfully pulled: ai/smollm2:360M-Q4_K_M
📋 Listing all available models...
📊 Found 1 models:
   1. {/* model details */}
🔍 Inspecting model: ai/smollm2:360M-Q4_K_M
✅ Model details: {/* inspection results */}
🎉 Tutorial completed successfully!
🧹 Cleaning up container...
✅ Container terminated successfully

Step 7: Best Practices

7.1 Resource Management

// Always use context with timeout for long operations
ctx, cancel := context.WithTimeout(context.Background(), 300*time.Second)
defer cancel()

// Always defer container cleanup
defer func() {
    if err := testcontainers.TerminateContainer(dmrCtr); err != nil {
        log.Printf("failed to terminate container: %s", err)
    }
}()

7.2 Error Handling

// Implement retry logic for network operations
func pullWithRetry(dmrCtr *dockermodelrunner.Container, ctx context.Context, modelName string, maxRetries int) error {
    for i := 0; i < maxRetries; i++ {
        if err := dmrCtr.PullModel(ctx, modelName); err != nil {
            if i == maxRetries-1 {
                return err
            }
            time.Sleep(time.Duration(i+1) * 5 * time.Second)
            continue
        }
        return nil
    }
    return nil
}

7.3 Model Naming Convention

Use the format: namespace/name:tag
Examples:
- ai/smollm2:360M-Q4_K_M
- ai/llama3.2:latest
- huggingface/bert:base-uncased

Troubleshooting

Common Issues

Container fails to start: Ensure Docker is running and you have sufficient resources
Model pull timeout: Increase timeout duration or check internet connection
Permission errors: Ensure your user has Docker permissions

Debug Tips

// Enable verbose logging
log.SetFlags(log.LstdFlags | log.Lshortfile)

// Add debug prints
log.Printf("Debug: Current context deadline: %v", ctx.Deadline())

Next Steps

Explore the Docker Hub AI models collection
Learn about Models as OCI Artifacts
Check the Testcontainers Go documentation

Conclusion

You’ve now learned how to:

Set up and use the Docker Model Runner with Testcontainers Go
Pull, inspect, and list AI models as OCI artifacts
Handle errors and timeouts properly
Implement best practices for resource management

The Docker Model Runner module makes it easy to work with AI models in your Go applications using the familiar Testcontainers approach.

References

How to successfully run Open WebUI with Docker Model Runner