How to Create Kubernetes Operators With Kubebuilder

Introduction

Kubernetes Operators automate the management of complex applications running on Kubernetes. Operators extend the functionality of Kubernetes by managing custom resources alongside core Kubernetes resources like Pods, Deployments, and Services. Operators can handle installation, upgrades, configuration, scaling, and monitoring tasks automatically.

Kubebuilder is a powerful framework for building Kubernetes Operators. It simplifies operator development by providing boilerplate code, scaffolding, and a clear structure for creating and managing custom resources.

In this tutorial, we’ll walk through the steps to create a Kubernetes Operator using Kubebuilder.

Prerequisites

Before you begin:

1. Install Go (v1.13 or later).
2. Install kubectl for interacting with your Kubernetes cluster.
3. Install Docker for container image builds.
4. Set up a Kubernetes cluster (e.g., using minikube or kind).
5. Install Kubebuilder on your local machine.

Installing Kubebuilder

Kubebuilder is not available via default package managers, so you need to install it manually. Follow these steps to install Kubebuilder on your system.

1. Download the latest release of Kubebuilder.

curl -L https://github.com/kubernetes-sigs/kubebuilder/releases/download/v3.4.1/kubebuilder_linux_amd64.tar.gz -o kubebuilder.tar.gz

2. Extract the tarball and move the binary to /usr/local.

tar -zxvf kubebuilder.tar.gz
sudo mv kubebuilder /usr/local/kubebuilder
export PATH=$PATH:/usr/local/kubebuilder/bin

3. Verify the installation.

kubebuilder version

Expected output:

Version: version.Version{KubeBuilderVersion:"v3.4.1", KubernetesVendor:"1.23.0", GoVersion:"go1.17.1", Compiler:"gc", Platform:"linux/amd64"}

Creating a New Kubebuilder Project

With Kubebuilder installed, you can now scaffold your Operator. Kubebuilder generates all necessary files and directories to kickstart development.

1. Create a new directory for your project:

mkdir my-operator
cd my-operator

2. Initialize the Kubebuilder project:

kubebuilder init --domain mydomain.com --repo github.com/myrepo/my-operator

• --domain specifies the API group domain.
• --repo sets the module path for the Go project.
3. After initializing, the project structure will resemble the following:

.
├── api
├── config
├── controllers
├── Dockerfile
└── go.mod

Creating an API and Controller

Next, generate an API and controller for your custom resource (CR). In this example, we’ll create a custom resource named App under the group apps.mydomain.com and version v1alpha1.

1. Create the API and controller.

kubebuilder create api --group apps --version v1alpha1 --kind App

2. When prompted, choose y for both options to generate the resource and the controller.

Create Resource [y/n]
Create Controller [y/n]

3. This command creates several new files.
• api/v1alpha1/app_types.go: Defines the App CRD.
• controllers/app_controller.go: Handles the reconciliation loop for the App resource.

Defining the Custom Resource

Now that we’ve generated the CRD, let’s define the structure of our App custom resource in app_types.go.

1. Open api/v1alpha1/app_types.go and modify the AppSpec struct to include the desired fields.

type AppSpec struct {
    Replicas int32 `json:"replicas"`
    Image    string `json:"image"`
}

2. The Replicas field represents the number of replicas of the application, and the Image field represents the container image.
3. After modifying the CRD, run the following command to regenerate the CRD manifests:

make manifests

Implementing the Controller Logic

Controllers in Kubernetes are responsible for managing the lifecycle of custom resources. In our App operator, we want the controller to create a Deployment for the App resource based on the custom resource’s specification.

1. Open the file controllers/app_controller.go and locate the Reconcile function. This function defines how the controller reacts to changes in the App resource.
2. Modify the Reconcile function to create or update a Deployment:

import (
    appsv1 "k8s.io/api/apps/v1"
    corev1 "k8s.io/api/core/v1"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
)

func (r *AppReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    // Fetch the App instance
    var app mydomainv1alpha1.App
    if err := r.Get(ctx, req.NamespacedName, &app); err != nil {
        return ctrl.Result{}, client.IgnoreNotFound(err)
    }

    // Define the desired Deployment
    dep := &appsv1.Deployment{
        ObjectMeta: metav1.ObjectMeta{
            Name:      app.Name,
            Namespace: app.Namespace,
        },
        Spec: appsv1.DeploymentSpec{
            Replicas: &app.Spec.Replicas,
            Selector: &metav1.LabelSelector{
                MatchLabels: map[string]string{"app": app.Name},
            },
            Template: corev1.PodTemplateSpec{
                ObjectMeta: metav1.ObjectMeta{
                    Labels: map[string]string{"app": app.Name},
                },
                Spec: corev1.PodSpec{
                    Containers: []corev1.Container{
                        {
                            Name:  "app-container",
                            Image: app.Spec.Image,
                        },
                    },
                },
            },
        },
    }

    // Set the App instance as the owner of the Deployment
    if err := controllerutil.SetControllerReference(&app, dep, r.Scheme); err != nil {
        return ctrl.Result{}, err
    }

    // Create or update the Deployment
    if err := r.Create(ctx, dep); err != nil {
        return ctrl.Result{}, err
    }

    return ctrl.Result{}, nil
}

3. This code defines a Deployment based on the App resource’s Replicas and Image fields. The controller creates or updates the Deployment whenever the App resource is modified.

Testing the Operator

With the controller logic implemented, we can test our operator by deploying it to a Kubernetes cluster.

1. Build and push the Docker image for the Operator.

make docker-build docker-push IMG=<your-registry>/my-operator:v0.1

2. Deploy the Operator to the Kubernetes cluster.

make deploy IMG=<your-registry>/my-operator:v0.1

3. Check that the operator is running in your cluster.

kubectl get pods -n my-operator-system

4. Apply the CRD for the App resource.

kubectl apply -f config/samples/apps_v1alpha1_app.yaml

5. Verify that the operator creates the corresponding Deployment.

kubectl get deployments

Expected output:

NAME              READY   UP-TO-DATE   AVAILABLE   AGE
app-deployment    1/1     1            1           2m

Conclusion

In this tutorial, we walked through the process of creating a Kubernetes Operator using Kubebuilder. Operators allow you to automate the management of complex applications on Kubernetes by managing custom resources. We covered the basics of Kubebuilder, including initializing a project, creating an API and controller, defining custom resources, and writing the reconciliation logic for our custom resource.

For more advanced topics such as implementing webhooks or adding validations to custom resources, refer to the official Kubebuilder documentation.

Resources

• Kubernetes – Collabnix
• KubeBuilder tutorial – Reddit
• KubeBuilder – Github