In the last article we saw how to run Jenkins Docker in Docker Agent. In addition to its obvious use case of building and pushing Docker images, I also noted that such an agent allows to run
minikube inside. Today, I’ll show how to run
Docker container. Of course, we’ll use
docker in docker client and daemon images for that.
If all you need is step by step procedure for running
Docker container, jump straight to the demo. Otherwise, if you want to understand the motivation better, keep reading.
Docker in Docker Jenkins agent is great for
Jenkins which is running in
Kubernetes and provisions its agents on demand. We also saw such an agent main use case – building
docker images and pushing them to some registry. That’s cool. However, what if you want to take your CI/CD to the next level and deploy helm charts as part of the release pipeline.
Deploy and test helm charts as part of CI/CD
Why would you want to do that? For example, for testing chart’s successful deployment before you release it i.e push it to some registry. In addition, you may want to run some tests after deployment, e.g. acceptance tests triggered my magic command
helm test. You can, of course, skip these steps, push the probably not working chart and know about it only when it reached production. Even if the deployment succeeds, but some basic sanity test fails after deployment, it’s bad, right? Yes, it’s better to have some automated test running before the actual release. This test will run on a successfully deployed chart in CI/CD environment during release pipeline.
What is this environment? There are many options. Today, I’ll show the option of
minikube cluster started in
docker container. We’ll deploy the chart there and run acceptance tests using
helm test. That will give the idea of what could run during release of helm charts if the same commands will run in Docker in Docker Jenkins agent.
Minikube in Docker container options
We’ll see 2 options:
minikube in Dockerclient in
Dockercontainer. This container will, of course, connect to another
Docker daemon in Dockercontainer
See Docker Client or Docker Daemon in Docker? if you are confused.
Enough talking. Let’s see these 2 options in action.
Minikube in Docker demo
Today, you’ll only need
Docker for running images.
minikube in Docker daemon container
It’s quite easy to run
docker daemon container. Let’s do that:
$ docker run --name dind -d --privileged docker:20.10.17- dind $ docker exec -it dind sh / # wget https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 / # mv minikube-linux-amd64 minikube / # chmod +x minikube / # ./minikube start --force ... * Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default / # ./minikube kubectl -- run --image=hello-world / # ./minikube kubectl -- logs pod/hello Hello from Docker! ...
minikube in Docker client container
--force flag starts
docker driver as
root. This is undesirable as
minikube warns us.
Let’s now run
docker client container which will connect to
docker daemon container.
Firstly, let’s start
docker daemon container as before, yet now let’s connect it to
docker network and give it network alias of
docker run --name dind -d --privileged --network dind --network-alias dind -e DOCKER_TLS_CERTDIR="" docker:dind
"" in order to not require
docker client to authenticate against the daemon. Inspect the logs for making sure the daemon started successfully or add health check which will do that for you.
$ docker logs -f dind ... time="2022-08-09T06:30:23.481053486Z" level=info msg="Daemon has completed initialization" time="2022-08-09T06:30:23.560717962Z" level=info msg="API listen on /var/run/docker.sock" time="2022-08-09T06:30:23.567636047Z" level=info msg="API listen on [::]:2375"
Now, let’s start
docker client container in the same network, enter its shell and make sure it can connect to the daemon:
docker run --name dind-client -it --network dind -e DOCKER_HOST="tcp://dind:2375"docker sh / # docker version ... Client: Version: 20.10.17 ... Server: Docker Engine - Community Engine: Version: 20.10.17
It works! Pay attention to
DOCKER_HOST environment variable which we set to the hostname and port of the deamon.
You can now repeat the same commands we ran in Docker daemon container in order to start
So, we started
docker client container as well. Now let’s put
minikube to some use by deploying helm chart to it and running some acceptance tests.
Deploy helm chart to minikube in docker and test it
- We’ll Install Jenkins helm chart.
# create dedicated namespace where Jenkins helm chart will be installed ./minikube kubectl create namespace jenkins # install helm wget https://get.helm.sh/helm-v3.9.2-linux-amd64.tar.gz tar -zxvf helm-v3.9.2-linux-amd64.tar.gz mv linux-amd64/helm /usr/local/bin/helm # install jenkins chart helm repo add jenkinsci https://charts.jenkins.io helm repo update helm install jenkins -n jenkins jenkinsci/jenkins
- After Jenkins images are downloaded and deployed (it may take several minutes) you will see below status:
./minikube kubectl -- get all -n jenkins pod/jenkins-0 2/2 Running 0 2m57s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S ) AGE service/jenkins NodePort 10.100.21.112 <none> 8080:3 0653/TCP 2m57s service/jenkins-agent ClusterIP 10.96.252.104 <none> 50000/ TCP 2m57s NAME READY AGE statefulset.apps/jenkins 1/1 2m57s
- Now, let’s run some acceptance test on deployed chart:
helm test jenkins -n jenkins NAME: jenkins LAST DEPLOYED: Tue Aug 9 06:59:11 2022 NAMESPACE: jenkins STATUS: deployed REVISION: 1 TEST SUITE: jenkins-tests Last Started: Tue Aug 9 07:04:05 2022 Last Completed: Tue Aug 9 07:04:05 2022 Phase: Succeeded TEST SUITE: jenkins-ui-test-ybu3s Last Started: Tue Aug 9 07:04:05 2022 Last Completed: Tue Aug 9 07:05:28 2022 Phase: Succeeded
Wow. That was cool. Now, imagine you would want to do the same in
CI/CD pipelines. Makes sense?
Alternatives? Ah, yes, there’s podman
Podman which uses the same CLI as
Docker. It’s indeed lightweight and doesn’t require a daemon
Docker needs. And it may be a good idea to run
minikube in a container with
podman installed. However, I didn’t succeed to run
minikube there. Moreover, from my understanding
podman is still not widely adopted. Have a look, for instance, at the number of questions at Stack Overflow tagged with
docker tags and make conclusions yourself.
That’s it about running
minikube in Docker container. You may also find useful my older
CI/CD articles useful. As always, feel free to share.
- Become a Certified Kubernetes Administrator (CKA)!
- Become a Certified Kubernetes Application Developer (CKAD)!
- BUNDLE KUBERNETES FUNDAMENTALS & CKA CERTIFICATION (COURSE & CERTIFICATION) FOR THE BEST DEAL! $499 ONLY!
Kubernetes courses on Pluralsight:
Sign up using this link to get exclusive discounts like 50% off your first month or 15% off an annual subscription)
Recommended Kubernetes books on Amazon: