1. Tainting Worker Nodes#

First, let’s taint our worker nodes:

1
kubectl taint nodes worker01 gpu=true:NoSchedule
2
kubectl taint nodes worker02 gpu=false:NoSchedule

These commands add taints to our worker nodes, effectively telling the scheduler not to place pods on these nodes unless they have matching tolerations.

2. Creating a Pod Without Tolerations#

Now, let’s create a simple nginx pod and see what happens:

1
apiVersion: v1
2
kind: Pod
3
metadata:
4
  name: nginx-pod
5
spec:
6
  containers:
7
    - name: nginx
8
      image: nginx

Save this as nginx-pod.yaml and apply it:

1
kubectl apply -f nginx-pod.yaml

Check the pod status:

1
kubectl get pods

You’ll notice that the pod is in a “Pending” state. Let’s investigate why:

1
kubectl describe pod nginx-pod

In the events section, you should see messages indicating that the pod can’t be scheduled due to the taints on the worker nodes.

3. Adding Toleration to the Pod#

Let’s modify our pod definition to include a toleration for the gpu=true:NoSchedule taint:

1
apiVersion: v1
2
kind: Pod
3
metadata:
4
  name: nginx-pod-with-toleration
5
spec:
6
  containers:
7
    - name: nginx
8
      image: nginx
9
  tolerations:
10
    - key: "gpu"
11
      operator: "Equal"
12
      value: "true"
13
      effect: "NoSchedule"

Apply this new configuration:

1
kubectl apply -f nginx-pod-with-toleration.yaml

Now, check the pod status again:

1
kubectl get pods -o wide

You should see that the pod is now scheduled on worker01, which has the matching taint.

4. Removing Taint from Control Plane Node#

By default, Kubernetes taints control plane nodes to prevent regular workloads from running on them. Let’s remove this taint:

1
kubectl taint nodes $(kubectl get nodes --selector=node-role.kubernetes.io/control-plane -o jsonpath='{.items[*].metadata.name}') node-role.kubernetes.io/control-plane:NoSchedule-

5. Creating a Redis Pod#

Now that we’ve removed the taint from the control plane, let’s create a Redis pod:

1
apiVersion: v1
2
kind: Pod
3
metadata:
4
  name: redis-pod
5
spec:
6
  containers:
7
    - name: redis
8
      image: redis

Apply this configuration:

1
kubectl apply -f redis-pod.yaml

Check where the pod is scheduled:

1
kubectl get pods -o wide

You should see that the Redis pod is now scheduled on the control plane node.

6. Re-adding Taint to Control Plane Node#

Finally, let’s add the taint back to the control plane node:

1
kubectl taint nodes $(kubectl get nodes --selector=node-role.kubernetes.io/control-plane -o jsonpath='{.items[*].metadata.name}') node-role.kubernetes.io/control-plane=:NoSchedule

Conclusion#

Through these exercises, we’ve explored the practical application of taints and tolerations in Kubernetes. We’ve seen how taints can prevent pods from being scheduled on specific nodes, and how tolerations allow pods to bypass these restrictions.

Key takeaways:

1. Taints are applied to nodes and prevent pods from being scheduled unless they have matching tolerations.
2. Tolerations are defined in pod specifications and allow pods to be scheduled on nodes with matching taints.
3. Control plane nodes are typically tainted by default to prevent regular workloads from running on them.
4. Removing taints from control plane nodes allows regular pods to be scheduled there, which can be useful in development or testing scenarios.

Remember, while taints and tolerations provide powerful control over pod scheduling, they should be used judiciously. In production environments, it’s generally recommended to keep the control plane dedicated to system tasks and schedule application workloads on worker nodes.

As you continue your Kubernetes journey, experiment with different combinations of taints and tolerations to see how they affect pod scheduling in your cluster. Happy Kuberneting!