Share the love

Taints and tolerations are two important concepts in Kubernetes that allow you to control how pods are scheduled on nodes. Taints are used to repel pods from nodes, while tolerations are used to attract pods to nodes. Together, they provide a powerful mechanism for fine-grained pod placement control.

In this guide, we will cover what taints and tolerations are, how they work, and how to use them in your Kubernetes clusters.

What are Taints and Tolerations?

Taints are attributes that you can apply to nodes in a Kubernetes cluster. They indicate that a node should not be used for certain types of workloads, for example, a node that is running a specialized application that should not be disturbed. When a pod is created, it is scheduled on a node that does not have any taints that the pod cannot tolerate.

Tolerations, on the other hand, are attributes that you can apply to pods. They indicate that a pod can tolerate certain taints on a node and should be scheduled on such nodes. Tolerations allow you to schedule pods on tainted nodes that would otherwise be unschedulable.

How Taints and Tolerations Work

When a pod is created, the Kubernetes scheduler checks the taints on each node and the tolerations on the pod. If there are no matching tolerations, the pod is not scheduled on the node.

When a pod is already running on a node and a new taint is added to the node, the pod is evicted if it does not have a toleration for the new taint.

Example

Let’s say you have a node that is running a specialized application that should not be disturbed. To ensure that no other pods are scheduled on this node, you can add a taint to the node like this:

kubectl taint nodes node1 special=special:NoSchedule

Now, when you try to create a new pod, it will not be scheduled on the node1 because it has the taint special=special:NoSchedule and the pod does not have a toleration for it.

To schedule a pod on this node, you must create a pod that tolerates the taint, or you must add a toleration to the pod’s configuration. This is done by adding a “tolerations” block to the pod’s YAML file, like so:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: my-image
  tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "special-user"
    effect: "NoSchedule"

In the above example, the pod is configured to tolerate a taint with the key “dedicated” and the value “special-user”. This means that the pod can be scheduled on a node that has that taint, even if no other pods can be scheduled there.

Another useful feature is toleration seconds, which allows a pod to tolerate a taint for a specific amount of time. For example:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: my-image
  tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "special-user"
    effect: "NoSchedule"
    tolerationSeconds: 3600

In this case, the pod will tolerate the taint for 3600 seconds (1 hour) before it is evicted.

It’s important to note that taints and tolerations are only effective if the pod scheduler is configured to take them into account. By default, Kubernetes scheduler does not take taints and tolerations into account, so you will have to configure your own scheduler or use a third-party scheduler that supports taints and tolerations.

Still didn’t understand?

Here’s another example to help understand Kubernetes taints and tolerations:

Imagine a group of coworkers, and one of them, let’s call him Bob, always brings in smelly food for lunch. Everyone in the office wants to avoid sitting near Bob because of the strong odor. This is similar to how pods in Kubernetes can be “tainted” with certain attributes, such as having a specific label or running on a specific node.

Now, let’s say that one of the coworkers, let’s call her Sally, is immune to the smell of Bob’s food. She can sit next to him without any issues. This is similar to how pods in Kubernetes can have “tolerations” for specific taints, allowing them to run on tainted nodes.

In this scenario, the office manager can “taint” Bob’s node with the “smelly food” taint, and then only allow pods with a “toleration” for the “smelly food” taint to run on that node. This way, only Sally’s pod will be scheduled to run on Bob’s node, while the rest of the team can work in a more pleasant environment.

Of course, in the real world of Kubernetes, the taints and tolerations would be applied using code and commands, rather than office managers and coworkers. But hopefully, this analogy helps make the concept more relatable.