Let's do some Karpenter work

Today, I will speak about a Karpenter, and it is not a grammatical error. I am not going to speak about craftsmen who build and repair wooden furniture. I will speak about a tool that can decrease your monthly bills for AWS or Azure cloud providers.

Karpenter is a framework used for Kubernetes cluster autoscaling. Currently, it only supports two cloud vendors, AWS and Azure. Both providers have their cluster auto-scale solutions, and it is raising questions, why we need Karpenter?

I will explain my experience with Karpenter on EKS (Amazon Managed Kubernetes Cluster). EKS is providing cluster autoscale through managed node groups, and it can be a good start, but as your cluster grows, you will notice that it is not flexible enough, and you will have more node groups to manage than you need.

There are several benefits of using Karpenter. One of them is speed. Karpenter is communicating directly to the EC2 API, which reduces network calls, so it takes around 1 minute to spin up a new EC2 instance and join it to the EKS cluster.

The second one is flexibility. Karpenter configuration file (I will explain details later in the text) can be set to use a node with really specific details.

Consolidation is one excellent Karpenter feature. This feature, if set, will scan your nodes and, if some of them are underutilized, it will rearrange Kubernetes pods to maximize the usage of current nodes. This can decrease resources in your cluster, and that means it will also decrease your bill. Because Karpenter has access to the AWS instance costs API, if it calculates that it is cheaper to create a bigger EC2 instance and move all pods there, it will do it automatically.

One note is that you need to set resources on your Kubernetes deployment, so Karpenter can decide what Node size is the best for your workload.

Installation

There are different ways to install Karpenter. It can be done from Terraform if you use it for IAC.

You can also follow the official documentation from Karpeneter.

Configuration

After we install Karpenter, we need to deploy two resource types in our EKS cluster.

We need to define NodePool and EC2NodeClass.

The node pool will define and filter the exact type of EC2 node we need. We can choose CPU architecture, OS, capacity type, node family, instance generation, region etc. You can set a limit on resource usage in the pool and choose when to do consolidation and for what reason.

More options can be read here.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: default
spec:
  template:
    spec:
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: kubernetes.io/os
          operator: In
          values: ["linux"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["2"]
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      expireAfter: 720h # 30 * 24h = 720h
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m

Image 1 - NodePool configuration

EC2NodeClass is going to define specific AWS EC2 settings in terms of what AMI we are going to use, subnet, tags, security groups...

Node Classes enable configuration of AWS specific settings. Each NodePool must reference an EC2NodeClass using spec.template.spec.nodeClassRef. Multiple NodePools may point to the same EC2NodeClass.

More can be read here.

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  role: "KarpenterNodeRole-${CLUSTER_NAME}" # replace with your cluster name
  amiSelectorTerms:
    - alias: "al2023@${ALIAS_VERSION}"
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: "${CLUSTER_NAME}" # replace with your cluster name
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: "${CLUSTER_NAME}" # replace with your cluster name	

Image 2 - EC2NodeClass configuration

After you have set Karpenter with the NodePool and EC2NodeClass, you need to specify a Kubernetes deployment to deploy resources on the Karpenter node using nodeSelector or tolerations...

	  nodeSelector:
    	karpenter.sh/nodepool: spot

Image 3 - nodeSelector example

  tolerations:
    - key: "type"
      operator: "Equal"
      value: "spot"
      effect: "NoSchedule"

Image 4 - Kubernetes tolerations example

In image 3, you see that spot instances are used. Spot instances are unused EC2 capacity in the AWS cloud and are available at up to a 90% discount compared to On-Demand prices. This is a significant reduction in your bill. The downside of spot instances is that AWS can reclaim them on demand, and this can put you at risk that you lose your Kubernetes resources. That is why it is not recommended to use them in production.

Karpenter is minimizing this risk because as soon as it receives notification that spot instances will be reclaimed, it will provision new ones (if they are available based on your NodePool configuration) and migrate all pods to new nodes in a cluster, so you don't need to worry about that.

Spot instances can be used for different batch tasks or for specific apps which are not need to be up 24/7.

If you want to visualize Karpenter nodes and CPU/RAM usage on those nodes, there is a small utility that can help you with that. It is called EKS node viewer.

This utility can also show you the current cost for your cluster.

Image 5 - EKS node viewer

If you use EKS or AKS, don't miss a chance to test Karpenter. You can reduce your cloud cost, and depending on your cluster size, it can be a lot of green bills at the end of the month.