Kubernetes networking might seem like a service we get for free from
Kubernetes. Yet, it’s important to understand the principles and ideas behind it. Keep reading to find them out.
Kubernetes networking main design decisions
Firstly, let’s state important design decisions Kubernetes developers took:
- pods may reach pods on any node using a real ip address, there’s no NAT.
kubeletcan reach all pods on a node.
- pods and services may reach each other using DNS name
There are different Kubernetes networks which implement the above design:
- node network – network of physical or virtual machines spanning Kubernetes cluster.
- pod network – pod ips may be taken from the node network. Yet, usually, the ips are takend from a pool of ips – CIDR range.
- cluster network – ips for services of type
clusterIPare assigned in this network.
Kubernetes pods communications
When we explored Jenkins Docker in Docker Agent, we saw an example of
Kubernetes pod that runs more than one container. How did they communicate? Right, using
What about pod to pod communication? It Depends. If Pods run on the same node, they will reach each other using their ip addresses with the help of some network interface. The interface might be a software bridge or tunnel. It depends on the chosen network model.
Whereas, if pods run on different nodes, nodes network must provide connectivity between pods using their ips. This is provided either using by L2/L3 or an overlay network. Overlay network gives an illusion that nodes are connected using a single network regardless of underlying network infra.
CNI is Container Network Interface. It defines container networking for multiple container orchestrators e.g.
Kubernetes. Kubernetes decided that its admins have to choose CNI plugin (e.g. Calico) which implements Kubernetes networking according to their needs. Such network plugin bridges between
Kubernetes and container runtime (e.g. docker) along with Kubernetes node OS for the goal of managing networking resources (e.g. ip addressing)
kubelet node local network configuration defines network plugin. It may either be CNI or
Kubenet. Let’s now see a demo on
Kubernetes networking demo
I assume you have
Kubernetes cluster. If you don’t, install on your machine
Kubernetes multi-node cluster using command:
minikube start --nodes 3 -p multinode-demo.
Kubernetes CNI demo
kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME munode Ready control-plane 9m55s v1.25.2 192.168.103.2 <none> Ubuntu 20.04.5 LTS 4.15.0-193-generic docker://20.10.18 munode-m02 Ready <none> 9m12s v1.25.2 192.168.103.3 <none> Ubuntu 20.04.5 LTS 4.15.0-193-generic docker://20.10.18 munode-m03 Ready <none> 8m48s v1.25.2 192.168.103.4 <none> Ubuntu 20.04.5 LTS 4.15.0-193-generic docker://20.10.18
Let’s deploy sample
Kubernetes deployment to the cluster. It’ll be the same one we used in this demo.
kubectl create deploy nginx --image=rancher/hello-world --replicas=3
You see each pod, its ip and node it’s running on:
kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-5844775f89-4nnwf 1/1 Running 0 14s 10.244.0.3 munode <none> <none> nginx-5844775f89-9ptln 1/1 Running 0 14s 10.244.2.2 munode-m03 <none> <none> nginx-5844775f89-r6tkk 1/1 Running 0 14s 10.244.1.2 munode-m02 <none> <none>
You wonder how traffic from a pod running on node
munode will reach other pod running on node
There’s a great explanation about
kindnet network plugin used by
Pay attention that if you want to inspect node networking configuration you can run the commands in below privileged shell:
kubectl debug node/[node-name] -it --image=mcr.microsoft.com/aks/fundamental/base-ubuntu:v0.0.11
Now you can run
ip route and
ip route get [some_pod_ip] commands. This is based on this trick. Of course, you can also connect to the node using
minikube ssh. But then, you may have to install some inspection binaries.
That’s it about Kubernetes networking. As always, feel free to share.
- Become a Certified Kubernetes Administrator (CKA)!
- Become a Certified Kubernetes Application Developer (CKAD)!
- BUNDLE KUBERNETES FUNDAMENTALS & CKA CERTIFICATION (COURSE & CERTIFICATION) FOR THE BEST DEAL! $499 ONLY!
Kubernetes courses on Pluralsight:
- Configuring and Managing Kubernetes Networking, Services, and Ingress
- Managing the Kubernetes API Server and Pods
Sign up using this link to get exclusive discounts like 50% off your first month or 15% off an annual subscription)
Recommended Kubernetes books on Amazon: