What’s New in Kubernetes 1.29: PersistentVolume Access Mode, Node Volume Expansion, KMS Encryption, Scheduler Optimization, and More

Kubernetes 1.29 draws inspiration from the intricate art form of Mandala, symbolizing the universe’s perfection. This theme reflects the interconnectedness of the Kubernetes community, with each contributor weaving a vibrant tapestry of collaboration.

Graduated to Stable:

ReadWriteOncePod PersistentVolume Access Mode:

In prior versions, the ReadWriteOnce access mode allowed multiple pods on the same node to read and write to the same volume.

Kubernetes 1.29 introduces ReadWriteOncePod as a stable feature, ensuring that a pod is the only one across the entire cluster that can read from or write to a PVC.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOncePod
  storageClassName: standard
  resources:
    requests:
      storage: 5Gi

Node Volume Expansion Secret Support for CSI Drivers:

CSI drivers may require secrets during node volume expansion, addressing scenarios like encrypted block storage or validations requiring credentials.
This feature became generally available, enhancing security and flexibility in storage operations.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast
provisioner: example.com/csi
secrets:
  - name: csi-driver-secret
    key: secret-key

KMS v2 Encryption at Rest:

Kubernetes 1.29 brings stability to Key Management Service (KMS) v2, offering improved performance, key rotation, health checks, and observability for encrypting API data at rest.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: encrypted
provisioner: kubernetes.io/aws-ebs
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

Graduated to Beta:

QueueingHint Feature for Scheduler Throughput:

Aiming to optimize scheduling efficiency, this beta feature introduces QueueingHint to reduce unnecessary scheduling retries significantly.

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  schedulerName: custom-scheduler
  queueingHint: HighThroughput

Node Lifecycle Separated from Taint Management:

Decouples TaintManager from NodeLifecycleController, separating node tainting from pod eviction and enhancing cluster resilience.

apiVersion: v1
kind: Node
metadata:
  name: my-node
spec:
  taints:
    - key: example.com/special
      effect: NoExecute
      value: "true"

Clean Up for Legacy Secret-based ServiceAccount Tokens:

The LegacyServiceAccountTokenCleanUp beta feature labels and removes unused legacy secret-based tokens, reducing potential security risks.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-service-account
automountServiceAccountToken: false

New Alpha Features:

Define Pod Affinity or Anti-affinity Using matchLabelKeys:

Enhances PodAffinity/PodAntiAffinity with matchLabelKeys for more accurate calculations during rolling updates.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  template:
    spec:
      affinity:
        podAffinity:
          matchLabelKeys:
            - app

nftables Backend for kube-proxy:

Introduces nftables as an alternative backend for kube-proxy, addressing performance issues and aligning with Linux distribution trends.

apiVersion: v1
kind: ConfigMap
metadata:
  name: kube-proxy-config
data:
  config.conf: |
    featureGates:
      SupportNftablesBackend: true

APIs to Manage IP Address Ranges for Services:

Allows dynamic management of IP ranges for Services, introducing ServiceCIDR and IPAddress API objects.

apiVersion: networking.k8s.io/v1
kind: ServiceCIDR
metadata:
  name: my-service-cidr
spec:
  cidr: 10.0.0.0/24

Image Pull Per Runtime Class:

Adds support to pull different container images based on the RuntimeClass of the Pod, providing flexibility for diverse runtime environments.

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  runtimeClassName: my-runtime-class
  containers:
    - name: my-container
      image: my-runtime-class-image

In-place Updates for Pod Resources for Windows Pods:

Extends the alpha feature for in-place updates of Pod resources to Windows containers, allowing resource changes without Pod restarts.

apiVersion: v1
kind: Pod
metadata:
  name: my-windows-pod
spec:
  containers:
    - name: my-container
      resources:
        limits:
          memory: "2Gi"
        requests:
          cpu: "500m"

Deprecations and Removals:

Removal of In-tree Integrations with Cloud Providers (KEP-2395):

Kubernetes 1.29 defaults to operating without built-in integrations with cloud providers.
Users can enable external cloud controller managers or opt back in to the legacy integration with associated feature gates.

kubeadm init --cloud-provider=external

Removal of v1beta2 Flow Control API Group:

The v1beta2 API version of FlowSchema and PriorityLevelConfiguration is no longer served in Kubernetes 1.29.

Users are advised to migrate to v1beta3 API version.

apiVersion: flowcontrol.apiserver.k8s.io/v1beta3
kind: FlowSchema

Deprecation of status.nodeInfo.kubeProxyVersion Field for

Node:

The .status.kubeProxyVersion field for Node objects is deprecated, with plans to remove it in a future release.

apiVersion: v1
kind: Node
metadata:
  name: my-node
status:
  kubeProxyVersion: deprecated

Legacy Linux Package Repositories:

In August 2023, legacy package repositories (apt.kubernetes.io and yum.kubernetes.io) were deprecated, and community-owned repositories (https://pkgs.k8s.io) were introduced for Debian and RPM packages.

Conclusion:

Kubernetes 1.29 unfolds a universe of possibilities, with stable features offering enhanced persistence, storage flexibility, and encryption capabilities. Beta features optimize scheduling throughput and streamline node lifecycle management, while alpha features introduce finer control over pod affinity, alternative proxy backends, and dynamic IP address range management.

As Kubernetes continues to evolve, the Mandala theme encapsulates the collaborative spirit of its community, creating a cosmic tapestry of innovation. For a detailed exploration of all changes, refer to the official release notes and embrace the future of container orchestration with Kubernetes 1.29.

Further Reading