How To Monitor Kubernetes Health With Probes

Table of Contents

Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. It ensures that your applications run smoothly and efficiently, but keeping track of their health is essential to maintaining their reliability and performance. Kubernetes uses probes to monitor the health of your applications and make sure they are functioning as expected.

This guide explains how to monitor the health of your Kubernetes applications using different types of probes. You will learn how to set up liveness, readiness, and startup probes to ensure your applications remain healthy and responsive. Whether you are new to Kubernetes or looking to improve your monitoring practices, this article will help you keep your applications running reliably.

Types of Probes

In Kubernetes, the three types of probes – Readiness, Liveness, and Startup Probes – play a critical role in managing application health and readiness within the cluster. Each probe has a unique function, ensuring that applications run smoothly and respond appropriately to changes in their state and the environment.

Readiness Probe: Kubernetes uses the Readiness Probe to assess a container’s readiness to handle incoming traffic. This assessment influences the inclusion or exclusion of a pod in a service’s load balancer. A failed Readiness Probe results in the pod’s removal from the service’s endpoints, effectively preventing traffic flow until the readiness check is passed. This probe is particularly valuable during application startup and maintenance tasks, guaranteeing that only sound and prepared containers fulfill user requests.

Liveness Probe: The Liveness Probe serves as a crucial mechanism for ensuring the continued operation and responsiveness of containers within a Kubernetes environment. By periodically checking the health of containers, the Liveness Probe can detect and address situations where a container has entered a deadlock or is otherwise unable to function properly. Upon detecting such a failure, the Liveness Probe triggers the restart of the affected container, thereby maintaining the overall health and availability of the application.

Startup Probe: The Startup Probe is used to check if an application within a container has started correctly. This probe is particularly useful for applications that have a long initialization phase. The Startup Probe runs once during container startup, and if it succeeds, it ensures that the container is in a proper state to run. If the Startup Probe fails, Kubernetes will restart the container. This helps in distinguishing between a container that is still starting and one that has failed to start properly, preventing premature restarts by the Liveness Probe during long startup periods.

Each probe can be configured using different methods, such as HTTP requests, TCP socket checks, or executing specific commands. Properly configuring these probes can help ensure the high availability, reliability, and stability of applications running in a Kubernetes environment.

Prerequisites

To follow this guide:

A Kubernetes Cluster
An Ubuntu Server to manage your cluster
Install Kubectl to access the cluster

Setting Up the Liveness Probe

Starting the Liveness Probe involves defining a liveness command and then making an HTTP request so that Kubernetes can check the health of the application. To set a Liveness Probe, use the following steps, as outlined below:

Using a text editor like vim, create a new yaml file called flask-live.yaml.
```
vim flask-live.yaml
```

Add the following configuration to the file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: flask-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: flask-app
  template:
    metadata:
      labels:
        app: flask-app
    spec:
      containers:
      - name: flask-app
        image: <your-dockerhub-username>/your-flask-app-image
        ports:
        - containerPort: 5000
        livenessProbe:
          httpGet:
            path: /check-health
            port: 5000
          initialDelaySeconds: 15
          periodSeconds: 5

This configuration setting will create a Kubernetes deployment object for a Flask application. The deployment is configured to launch three replica pods, each running the Docker image specified by “your-dockerhub-username/your-flask-app-image”.

Each pod contains a container running the Flask application, which listens on port 5000. The deployment includes a liveness probe to monitor the Flask app’s health. This probe sends HTTP GET requests to the “/check-health” endpoint on port 5000, starting 15 seconds after the pod launches and repeating every 5 seconds.

If the Flask application becomes unresponsive and the liveness probe fails, Kubernetes will take corrective action by automatically restarting the container. This ensures the application remains available and minimizes downtime, helping to maintain the overall reliability of the service.

Execute the following command to start this deployment.
```
kubectl apply -f flask-live.yaml
```
Check if the deployment is running.
```
kubectl get deployment
```
View pods created by deployment.
```
kubectl get pods
```
To expose the flask-app deployment through a Kubernetes service object to allow HTTP GET requests to the pods. Create a file flask-service.yaml.
```
vim flask-service.yaml
```

Add the following configuration code to the file.

apiVersion: v1
kind: Service
metadata:
  name: flask-service
spec:
  selector:
    app: flask-app
  type: NodePort
  ports:
    - port: 3000
      targetPort: 3000
      nodePort: 30000

This configuration defines a Kubernetes service for a Flask application. The service is named “flask-service” and uses a selector to target pods with the label “app: flask-app.” It is configured as a NodePort type, which makes the service accessible outside the Kubernetes cluster by exposing it on a specific port on each node.

The service forwards traffic from port 3000 to port 3000 on the target pods and maps it to node port 30000. This means that accessing any node in the cluster on port 30000 will route the traffic to port 3000 on the Flask application pods.

Create the service with kubectl.
```
kubectl apply -f flask-service.yaml
```
Check to verify if the service is running.
```
kubectl get services
```

Now send a substantial amount of GET requests to the “/check-health” route using a load testing tool like Siege.

Create a new file named siege-get.yaml.
```
vim siege-get.yaml
```

Add the following configuration.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: siege
spec:
  replicas: 1
  selector:
    matchLabels:
      app: siege
  template:
    metadata:
      labels:
        app: siege
    spec:
      containers:
      - name: siege
        image: dockersec/siege
        command: ["siege", "-c40", "-t1M", "http://10.245.221.142:3000/"]

The code above creates a Kubernetes deployment object with the dockersec/siege image, created for the Siege load testing tool.

spec: Describes the desired state of the Deployment.

replicas: Sets the number of pod replicas to be deployed to 1.
selector: Matches the labels of the pods to be managed by this Deployment.
template: Defines the pod template

spec: Describes the pod specification.

containers: Lists the containers to be deployed in the pod.
name: The name of the container is “siege”.
image: Specifies the Docker image “dockersec/siege”.
command: Executes the “siege” command with parameters to simulate 40 concurrent users for a duration of 1 minute against the specified URL.

The configuration sets Siege to create 40 concurrent users -c 40 to run a load test for 1 minute -t 1M on the specified URL http://10.245.221.142:3000/.

Create the deployment.
```
kubectl apply -f siege-get.yaml
```
Check to verify if the deployment is running
```
kubectl get deployments
```

You can frequently monitor the status of the liveness probes by using the get command.

kubectl get pods

As you continue to check the status of the liveness probes, you will notice that the count for the number of pods that get restarted will start to increase over time. This means that liveness is telling Kubernetes to restart pods that are unresponsive.

Setting Up the Readiness Probe

To set up a readiness probe, first ensure that your application has an endpoint to check its readiness state. Next, choose the probe type (HTTP, TCP, or Command). Specify the path, port, or command to be used. To set a Readiness Probe, use the following steps, as outlined below:

For this guide, we will use a sample Python application to simulate a cache operation for 27 seconds upon startup.

Using Vim, create a new file test-cache.yaml.
```
vim test-cache.yaml
```

Add the following configuration to the file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: python-testcache-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: python-testcache-app
  template:
    metadata:
      labels:
        app: python-testcache-app
    spec:
      containers:
      - name: nodejs-app
        image: <your-dockerhub-username>/test-cache
        ports:
        - containerPort: 3000
        readinessProbe:
          httpGet:
            path: /ready
            port: 3000
          initialDelaySeconds: 25
          periodSeconds: 13

This code creates two replicas of the application python-testcache-app running the application in a container and listening on port 3000. This configuration makes a GET request to the /ready endpoint of the cache application every 13 seconds and waits for another 25 seconds before starting the first probe.

Create the deployment.
```
kubectl apply -f test-cache.yaml
```
Check if the deployment is running.
```
kubectl get deployments
```
Next, view the pods that have been created by this deployment to check their readiness status.
```
kubectl get pods
```

Setting Up the Startup Probe

The Startup Probe configuration, similar to the previous probes we deployed, starts by specifying the probe type (HTTP, TCP, or Command), defining the path, port, or command. Next, define the initial delay until the probe begins, the time between checks, and the failure threshold that shows the app is still starting. To set a Startup Probe, use the following steps as outlined below:

Create a new file with Vim, startup-probe.yaml.
```
vim startup-probe.yaml
```

Then add the following configuration to the file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: python-testcache-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: python-testcache-app
  template:
    metadata:
      labels:
        app: python-testcache-app
    spec:
      containers:
      - name: nodejs-app
        image: <your-dockerhub-username>/startup-probe
        ports:
        - containerPort: 3000
        startupProbe:
          httpGet:
            path: /
            port: 3000
          failureThreshold: 30
          periodSeconds: 1
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 30
          periodSeconds: 10

With this Kubernetes Deployment, you set up a configuration where two pods are created. Each pod will run the Python application that is pulled from the Docker image located at <your-dockerhub-username>/startup-probe. These pods will be accessible on port 3000, where they are expected to operate and serve requests.

Each pod has a startup probe that sends HTTP GET requests to the path on port 3000. It runs every second until success or 30 failures. After startup, the liveness probe runs every 10 seconds, starting 30 seconds after the container starts. If the “/health” endpoint doesn’t respond with success, Kubernetes will kill and restart the pod after 3 failures.

Check that the deployment is running.
```
kubectl apply -f startup-probe.yaml
```
Check to verify if the pods created by the deployment have a ready state.
```
kubectl get deployments
```

Conclusion

By following this guide, you have successfully set up the three types of Probes — Liveness, Readiness, and Startup Probes — to effectively monitor the health of your Kubernetes containers. If you want to learn more about Kubernetes probes, visit the official documentation.