Kubecost 3.0 is Here: A Major Architectural Shift for Kubernetes Cost Management

KubeCost 3.0: Transforming Kubernetes Cost Management

Kubernetes cost management just got a serious upgrade. The Kubecost Helm Chart repository, which provides the templates for developing this enterprise-grade application to monitor and manage Kubernetes spend, has released version 3.0, introducing fundamental changes aimed at improving performance, granularity, and efficiency.

If you are currently running Kubecost 2.x, read on—the migration requires careful planning due to a significant architectural overhaul.

The Architecture Revolution: ClickHouse and Direct Metrics

Kubecost 3.0 represents a substantial evolution in how cost metrics are handled, fundamentally changing the underlying database and data ingestion mechanism.

The transition from using DuckDB and Prometheus in 2.x to leveraging ClickHouse and Direct metrics sourcing in 3.0 offers powerful performance benefits.

Crucial Migration Requirement: Data Re-ingestion

The switch to the ClickHouse database necessitates a complete re-ingestion of all historical data as soon as Kubecost 3.x is installed.

• Duration: This re-ingestion process can take a significant amount of time, ranging from 20 minutes to 2 days. The duration depends heavily on the performance of the storage backing the Persistent Volume and the overall dataset size.

• UI Status: During the re-ingestion, the User Interface (UI) will remain available, but users will see a progress indicator.

• Data Order: Data is imported starting from the present day and moving backwards in time until the full historical record is available.

Introducing the FinOps Agent (and its Benefits)

Alongside the database shift, Kubecost 3.0 introduces a powerful new component: the finops-agent. This agent replaces the previous cost-model container.

The new agent provides major enhancements for better resource optimization:

1. 10 Minute Metric Granularity: This level of detail is critical for accurate cluster/node group/container right-sizing.

2. Efficiency Gains: The new agent uses up to 50% less memory.

3. Enhanced Data: It provides more diagnostics data than its predecessor.

The Upgrade Path: Moving from 2.x to 3.0

The recommended procedure for upgrading from Kubecost 2.x to 3.x is conservative to ensure a smooth transition:

1. Intermediate Step (2.9): Update all existing agents to version 2.9.

2. Stabilization Period: Maintain these 2.9 agents for two days. Examples for how to upgrade to 2.9 are available in the GitHub repository.

3. Upgrade Primary: Finally, proceed with the upgrade to 3.0.

While the Kubecost 3.0 primary cluster is compatible with 2.x agents and newer, and the agent data is generally backwards compatible with Kubecost 2.x, mixing versions is not ideal for full visibility:

Getting Started and Kubernetes Support

The Kubecost Helm Chart repository is highly active, boasting 600 stars and 432 forks, and is licensed under the Apache-2.0 license. The latest noted release is v3.0.4.

Kubernetes Version Support: Kubecost 3.0 increases the minimum supported Kubernetes version to 1.29 (up to 1.34).

Installation: You can install the latest stable version easily using Helm:

helm install kubecost \
  --repo https://kubecost.github.io/kubecost kubecost \
  --namespace kubecost --create-namespace \
  --set global.clusterId=someclustername

If you wish to test pre-release features, you can install beta/release candidates by passing the --devel flag. The primary development is done on the develop branch, which is not stable and subject to change.

The move to 3.0 positions Kubecost to deliver faster, more precise, and more granular cost data than ever before, enabling better right-sizing and resource management within your Kubernetes environments.

For more details on the features and to explore the code, you can visit the Kubecost Helm chart repository on GitHub, or check the official documentation (IBM Docs)