As per the recent Cloud-Native AI report generated by CNCF during the KubeCon + CloudNativeCon Europe conference, AI is empowering operators and developers to work smarter, not harder. The convergence of Cloud Native methodologies with Artificial Intelligence (AI) has emerged as a transformative force reshaping industries and driving innovation. Using AI to improve cloud-native systems is no longer science fiction.
Source ~ https://www.cncf.io/wp-content/uploads/2024/03/cloud_native_ai24_031424a-2.pdf
Cloud Native technologies have revolutionized the way applications are developed, deployed, and managed in modern IT environments. By leveraging containerization, microservices architecture, and orchestration platforms like Kubernetes, organizations can achieve scalability, resilience, and agility in their operations. On the other hand, Artificial Intelligence and Machine Learning have paved the way for data-driven decision-making, predictive analytics, and automation across various domains.
Kubernetes, the container orchestration platform, is a powerful tool for managing applications in the cloud. But what if you could add some artificial intelligence muscle to your Kubernetes workflow? This is where OpenAI and these 3 open-source projects come in: Kubectl OpenAI client, K8sGPT, and KoPylot. These innovative tools leverage OpenAI’s capabilities to streamline tasks, automate processes, and gain deeper insights from your Kubernetes clusters.
Let’s dive in and explore how they can supercharge your Kubernetes experience.
1. KoPylot
KoPylot is a cloud-native application performance monitoring (APM) solution that runs on Kubernetes. It is designed to help developers and operations teams diagnose and troubleshoot issues in complex distributed systems. It provides real-time insights into application performance, including metrics, traces, and logs, to help teams identify and resolve issues quickly.
KoPylot is designed to help teams monitor and diagnose Kubernetes applications with ease. It provides a comprehensive view of application performance, including real-time metrics, traces, and logs, to help teams identify and resolve issues quickly.
Features
KoPylot provides a wide range of features to help teams monitor and diagnose Kubernetes applications, including:
- Real-time Metrics: KoPylot provides real-time metrics for Kubernetes workloads, including CPU, memory, and network usage. It also provides metrics for custom application-level metrics, which can be used to monitor specific application behaviors and performance.
- Distributed Tracing: KoPylot provides distributed tracing capabilities, allowing teams to trace requests across multiple microservices and identify bottlenecks and performance issues.
- Logs: KoPylot provides log aggregation capabilities, allowing teams to centralize logs from multiple containers and pods running on Kubernetes.
- Audit: KoPylot provides auditing capabilities, allowing teams to track changes to Kubernetes resources and monitor access to Kubernetes API server.
- Chat: KoPylot provides a chat interface, allowing teams to collaborate and share insights in real-time.
- Diagnose: KoPylot provides a diagnose feature, allowing teams to quickly identify issues and find potential solutions.
How it Works
KoPylot works by integrating with Kubernetes clusters and exposing a web-based interface that provides users with a comprehensive view of their workloads. KoPilot’s architecture consists of a set of microservices that communicate with each other over HTTP and gRPC.
When a user logs into KoPylot, they are presented with a dashboard that shows an overview of all the workloads running on their Kubernetes cluster. From the dashboard, users can drill down into individual workloads to view details such as resource utilization, status, and logs.
KoPylot communicates with Kubernetes clusters using the Kubernetes API server. The API server provides a standard interface for interacting with Kubernetes clusters and enables KoPilot to manage and monitor workloads running on any Kubernetes cluster, regardless of the underlying infrastructure.
KoPylot’s microservices are designed to be highly scalable and fault-tolerant. The microservices are deployed as Kubernetes pods and can be scaled horizontally as workload demand increases. Additionally, KoPilot’s microservices are designed to recover automatically from failures, ensuring that the system remains available and responsive at all times.
To diagnose issues in a Kubernetes cluster, KoPylot provides a diagnostic tool that can help identify problems quickly. The diagnostic tool checks for common issues and misconfigurations in the cluster and provides suggestions for resolving them. For example, it can check if any pods are not running, if there are any pending pods, or if any nodes are unreachable. The diagnostic tool can also help identify performance issues in the cluster by analyzing the resource usage of nodes and pods. It provides detailed information about CPU and memory usage, as well as network and disk I/O metrics.
In addition to monitoring and diagnosing issues, KoPylot also provides a chat interface that allows users to interact with the system using Natural Language. This feature enables users to ask questions and get answers about the status of the cluster, the configuration of specific resources, or any other aspect of the system.
Finally, KoPylot includes an audit log that records all actions taken by users within the system. This log can be used to trace changes made to the cluster and to identify any potential security issues.
The following is an example of using KoPylot to perform a health check on a service:
kopylot diagnose check -s my-service -n my-namespace
This command will run a series of checks on the specified service, including checking for pod readiness, HTTP status codes, and TCP connectivity. If any issues are found, KoPylot will provide a detailed report of the problem.
Installation
To install KoPylot on Kubernetes, you can follow these steps:
1. Requests an API key from OpenAI
Export the key using the following command:
export KOPYLOT_AUTH_TOKEN=your_api_key
2. Install KoPylot using pip:
# pip install kopylot
3. Run KoPylot
# kopylot --help
Usage: kopylot [OPTIONS] COMMAND [ARGS]...
╭─ Options ──────────────────────────────────────────────────────────────────────────╮
│ --version │
│ --install-completion [bash|zsh|fish|powershell Install completion for the │
│ |pwsh] specified shell. │
│ [default: None] │
│ --show-completion [bash|zsh|fish|powershell Show completion for the │
│ |pwsh] specified shell, to copy it │
│ or customize the │
│ installation. │
│ [default: None] │
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────╮
│ audit Audit a pod, deployment, or service using an LLM model. │
│ chat Start a chat with kopylot to generate kubectl commands based your │
│ inputs. │
│ ctl A wrapper around kubectl. The arguments passed to the ctl subcommand are │
│ interpreted by kubectl. │
│ diagnose Diagnose a resource e.g. pod, deployment, or service using an LLM model. │
╰────────────────────────────────────────────────────────────────────────────────────╯
Local Setup
If you prefer to set up your development environment locally, make sure you have Poetry installed on your system. Then, follow these steps:
Clone the KoPylot repository:
git clone https://github.com/avsthiago/kopylot
Navigate to the project folder
cd kopylot
Install the project dependencies using Poetry
make install
Real Workload Example
Now that we have covered the features and functionality of KoPylot, let’s take a look at a real workload example to see how it can be used in practice.
In this example, we will use KoPylot to diagnose a problem with a Kubernetes deployment.
Deploy a sample workload using the following YAML manifest:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
kubectl apply -f nginx-deployment.yaml
Run the following command to diagnose the deployment using KoPylot:
kopilot diagnose deployment nginx --namespace default
KoPylot will output the following result:
===========================================================================
Diagnosing Deployment nginx in namespace default
===========================================================================
---------------------------------------------------------------------------
Deployment nginx is running
---------------------------------------------------------------------------
Reason: The deployment is running correctly.
---------------------------------------------------------------------------
Deployment nginx is accessible
---------------------------------------------------------------------------
Reason: The deployment is accessible via the service.
---------------------------------------------------------------------------
Deployment nginx has enough resources
---------------------------------------------------------------------------
Reason: The deployment has enough resources.
---------------------------------------------------------------------------
Deployment nginx is not outdated
---------------------------------------------------------------------------
Reason: The deployment is using the latest available image.
===========================================================================
Diagnosis complete.
===========================================================================
This indicates that the deployment is running correctly and no issues were found.
Overall, KoPylot is a useful tool for diagnosing and troubleshooting Kubernetes workloads. Its web-based chat interface and CLI make it easy to use and accessible to all levels of users.
2. K8sGPT
K8sGPT is a tool that uses NLP to analyze logs and other data from Kubernetes clusters to identify and diagnose issues. It has a set of built-in analyzers that are designed to identify common issues such as pod crashes, service failures, and ingress misconfigurations. K8sGPT is built on top of OpenAI’s GPT-3 language model, which allows it to understand natural language and provide explanations that are easy to understand.
K8sGPT is focused on triaging and diagnosing issues in your cluster. It is a tool for SRE, Platform, and DevOps engineers to help them understand what is going on in their cluster and find the root cause of an issue. It can help you cut through the noise of logs and multiple tools to find the root cause of an issue.
How K8sGPT Works
K8sGPT uses a set of analyzers that are designed to identify issues in Kubernetes clusters. These analyzers use NLP to analyze logs, metrics, and other data from your cluster to identify potential issues. When an issue is identified, K8sGPT provides an explanation in natural language that is easy to understand. This allows you to quickly understand the issue and take the necessary steps to resolve it.
K8sGPT is built on top of OpenAI’s GPT-3 language model, which allows it to understand natural language. This means that you can ask K8sGPT questions about your cluster in plain English, and it will provide a response that is easy to understand. For example, you can ask K8sGPT “Why is my pod crashing?” and it will provide an explanation of why the pod is crashing and what steps you can take to fix the issue.
Installation
K8sGPT can be installed on Linux, Mac, and Windows. The easiest way to install K8sGPT on Linux or Mac is via Homebrew. To install K8sGPT via Homebrew, run the following commands:
brew tap k8sgpt-ai/k8sgptbrew install k8sgpt
If you encounter an error when installing K8sGPT on WSL or Linux, you may need to install the build-essential package. You can do this by running the following commands:
sudo apt-get updatesudo apt-get install build-essential
If you are running on Windows, you can download the latest Windows binaries from the Release tab on GitHub.
Once k8sgpt is installed, you can verify that it is working by running the version command:
k8sgpt version
Generating an API key
Before we can use k8sgpt, we need to generate an API key from OpenAI. To generate an API key, run below command , this will open a link in your default web browser where you can generate an API key. Once you have generated an API key, you can set it in k8sgpt
k8sgpt generateK8sgpt auth
This will prompt you to enter your API key. Once you have entered your API key, k8sgpt will be able to use it to analyze your Kubernetes clusters.
Analyzing your Kubernetes clusters
Now that we have k8sgpt installed and authenticated, we can start analyzing our Kubernetes clusters. To analyze a Kubernetes cluster, run the following command:
k8sgpt analyze
This will scan your Kubernetes cluster and look for any issues. By default, k8sgpt will use all of its built-in analyzers to analyze your cluster.
If k8sgpt finds any issues, it will output a summary of the issues it found. For example:
Analyzer: podAnalyzer
Namespace: default
Name: nginx-66b6c48dd5-zk6jt
Kind: Pod
Reason: CrashLoopBackOff
Message: Back-off 5m0s restarting failed container=nginx pod=nginx-66b6c48dd5-zk6jt_default(25f13c57-04eb-4a0a-a2f7-17b7564a7944)
Refer: https://kubernetes.io/docs/tasks/debug-application-cluster/debug-running-pod/
This output tells us that there is an issue with the nginx pod in the default namespace. The pod is in a CrashLoopBackOff state, and the nginx container in the pod has failed and is being restarted every 5 minutes.
If you want to see more information about the issue, you can use the --explain flag:
k8sgpt analyze --explain
This will output a more detailed explanation of the issues that k8sgpt found.
Filtering resources
By default, k8sgpt will analyze all resources in your Kubernetes cluster. However, you can filter the resources that k8sgpt analyzes by using the –filter flag.
Filters are used to control which Kubernetes resources are analyzed. By default, all filters are enabled. You can manage filters using the filters subcommand.
To view the list of filters, run:
Add filter , remove , list and more
k8sgpt filters list
k8sgpt filters add [filter]
k8sgpt filters add Service,Pod
k8sgpt filters remove [filter]
k8sgpt filters remove Service
You can filter the resources by namespace using the –namespace flag:
k8sgpt analyze --namespace=default
Custom Analyzers
k8sgpt comes with a set of built-in analyzers that cover a variety of Kubernetes resources. However, you can also write your own custom analyzers to analyze your own resources.
To write a custom analyzer, you will need to create a new Go package that implements the Analyzer interface:
type Analyzer interface {
// Analyze analyzes the given Kubernetes resource and returns any issues found
Analyze(resource *unstructured.Unstructured) ([ ]*Issue, error)
// Name returns the name of the analyzer
Name() string
// Enabled returns whether the analyzer is enabled
Enabled() bool
}
The Analyze method takes a Kubernetes resource as input and returns an array of issues found. The Name method returns the name of the analyzer, and the Enabled method returns whether the analyzer is enabled.
Once you have written your custom analyzer, you can register it with k8sgpt by calling the RegisterAnalyzer function:
RegisterAnalyzer(name string, analyzer Analyzer)
k8sgpt.RegisterAnalyzer("MyAnalyzer", &MyAnalyzer{})
You can then use the --filter flag to filter by your custom analyzer:
k8sgpt analyze --filter=MyAnalyzer
You can also combine multiple filters together to create more complex filters. For example, the following command will only analyze Pod resources in the “default” namespace with a label of “app=myapp”:
k8sgpt analyze --filter=Pod --namespace=default --label-selector=app=myapp
K8sGPT is a powerful tool that can help you diagnose and triage issues in your Kubernetes clusters. Its ability to analyze logs and Kubernetes resources using natural language processing and AI makes it stand out from other monitoring tools.
By installing and configuring K8sGPT, you can easily scan your clusters, identify issues, and get recommendations on how to fix them. Additionally, its built-in analyzers and filters make it easy to customize the analysis to fit your specific needs.Whether you’re an SRE, Platform or DevOps engineer, K8sGPT can help you gain insights into your Kubernetes clusters and make your job easier. Try it out today and see how it can help you improve your Kubernetes monitoring and troubleshooting workflows!
3. Kubectl OpenAI Client
The Kubectl OpenAI client project is a kubectl plugin to generate and apply Kubernetes manifests using OpenAI GPT.
Getting Started
- Install Docker Desktop
- Install Kubectl-ai
You can install it on your Macbook directly using Homebrew:
brew tap sozercan/kubectl-ai
https://github.com/sozercan/kubectl-ai
brew install kubectl-ai
Get OpenAI Keys
You can get the OpenAI keys from https://platform.openai.com/account/api-keys
Please Note: kubectl-ai requires an OpenAI API key or an Azure OpenAI Service API key and endpoint, and a valid Kubernetes configuration.
export OPENAI_API_KEY=<your OpenAI key>
Installing on CentOS
yum install wget
wget https://github.com/sozercan/kubectl-ai/releases/download/v0.0.10/kubectl-ai_linux_amd64.tar.gz
tar xvf kubectl-ai_linux_amd64.tar.gz
mv kubectl-ai /usr/local/bin/kubectl-ai
Setting up Kubeview
Using Helm
Assuming that you have already installed Git and Helm on your laptop, follow the below steps
git clone https://github.com/benc-uk/kubeview
cd kubeview/charts/
helm install kubeview kubeview
Testing it locally
kubectl port-forward svc/kubeview -n default 80:80
Deploying Pod using namespace
kubectl ai "Create a namespace called ns1 and deploy a Nginx Pod"
✨ Attempting to apply the following manifest:
---
apiVersion: v1
kind: Namespace
metadata:
name: ns1
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: ns1
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
Use the arrow keys to navigate: ↓ ↑ → ←
? Would you like to apply this? [Reprompt/Apply/Don't Apply]:
+ Reprompt
▸ Apply
Don't Apply
The YAML manifest you provided creates a basic Nginx pod with the name “nginx-pod” and exposes port 80. To apply this manifest and create the pod, you can use the kubectl apply command. Save the manifest in a file, for example, nginx-pod.yaml, and then execute the following command in your terminal:
Difference between “Create” and “Deploy” [Be Careful]
kubectl ai "Create a namespace called ns1 and create a Nginx Pod"
✨ Attempting to apply the following manifest:
apiVersion: v1
kind: Namespace
metadata:
name: ns1
---
apiVersion: v1
kind: Pod
metadata:
name: nginx
namespace: ns1
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
✔ Apply
Accessing the Nginx Pod via Web Browser
kubectl port-forward nginx 8000:80 -n ns1
Forwarding from 127.0.0.1:8000 -> 80
Forwarding from [::1]:8000 -> 80
Handling connection for 8000
Handling connection for 8000
Deployment
This is an example for deploying 3 replicas in the specific namespace
kubectl ai "create an nginx deployment with 3 replicas under namespace ns1"
✨ Attempting to apply the following manifest:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: ns1
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: webpage
image: ajeetraina/webpage
ports:
- containerPort: 80
✔ Apply
Services
kubectl ai "create an nginx deployment with 3 replicas under namespace ns1 and this time create service type as NodePort"
✨ Attempting to apply the following manifest:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: ns1
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: webpage
image: ajeetraina/webpage
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: ns1
spec:
type: NodePort
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 30080
Use the arrow keys to navigate: ↓ ↑ → ←
? Would you like to apply this? [Reprompt/Apply/Don't Apply]:
+ Reprompt
▸ Apply
Don't Apply
Listing the Kubernetes Resources
kubectl get po,deploy,svc -n ns1
NAME READY STATUS RESTARTS AGE
pod/nginx-deployment-58945458f5-5pk6b 1/1 Running 0 28s
pod/nginx-deployment-58945458f5-7htd7 1/1 Running 0 28s
pod/nginx-deployment-58945458f5-s6cxm 1/1 Running 0 28s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deployment 3/3 3 3 28s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx-service NodePort 10.100.230.251 <none> 80:30080/TCP 28sConclusion
The marriage of Kubernetes and OpenAI through these open-source projects is a game-changer for cloud-native management. Kubectl OpenAI client, K8sGPT, and KoPylot offer a powerful trio to streamline tasks, automate processes, and gain valuable insights from your clusters. Whether you’re a seasoned Kubernetes wrangler or just starting your journey, these AI-powered tools can help you tame the beast and achieve greater control over your cloud-native deployments. So, saddle up and get ready to experience a whole new level of efficiency and automation in your Kubernetes kingdom!
