Categories
Best Practices CI/CD DevOps

How to test helm charts?

As a developer of helm charts’ CI/CD pipelines you probably wondered how to test Helm Charts. This may seem challenging because on the one hand, it seems like there’s nothing to test. On the other hand, you probably encountered numerous cases when the deployment of helm chart failed because of some missing bracket. Keep reading to find out how to test helm charts. If you later find this article useful take a look at the disclaimer for information on how to thank me.

Table Of Contents
  1. Introduction
  2. Test helm charts as part of CI pipeline
  3. Test helm charts as part of Release pipeline
  4. Summary

Introduction

What are helm charts?

There are great resources on the web which explain what helm charts are. Hence, I won’t try to compete with them. I’ll just explain briefly what helm charts are, because keeping in mind this simple fact will simplify reasoning about testing helm charts and validating their correctness.

Helm charts, plainly, are a bunch of Kubernetes resources descriptors (yaml files) which comprise an application being deployed to Kubernetes cluster. The descriptors adhere to Kubernetes schema. That’s why, keys and their possible values in the descriptors have to be very specific. It’s also worth adding that some of the values inside the descriptors may be templated. The templates use values either from chart’s values.yaml or dynamically set on the server side during the chart’s deployment.

Note that helm charts may reference multiple container images which comprise the application (e.g. web service) and its different aspects (e.g. backup job).

Finally, the chart may be dependent on other charts which may provide infra services (e.g. db for web app)

I’m pretty sure, there’s a lot more to say about helm charts, but the above will be enough for defining helm chart correctness validations.

Why to test helm charts?

I believe that testing is a must for every piece of software. Moreover, these tests have to be automatic, cover different aspects and run within different CI/CD pipelines. Helm chart is not different. If we make even innocent typo in chart manifests, the deployment of a chart will fail. And this is too late in the SDLC process.

Test helm charts as part of CI pipeline

I’ll assume you are familiar with CI/CD terms. Read more about it in CI/CD using Jenkins and Docker if you need to.

Let’s see what helm chart tests we would like to run during CI pipeline. As it runs on each new pull request or its update, the tests have to be light and fast.

helm chart rendering validation

Once helm chart is deployed it’s first rendered on the server side. During the rendering Kubernetes manifests’ templates are replaced from different sources e.g. values.yaml. We don’t have to wait for deployment to test helm chart rendering. For local chart rendering, we can run helm template command. Yes, it won’t test for server side validity of the helm chart. However, it will provide a good indicator on the chart quality.

chart Kubernetes manifests syntax validation

This validation validates that the chart Kubernetes manifests adhere to Kubernetes schema. For instance, you can’t write anything you want in the manifests. If you describe a specific resource in yaml file, the fields in it have to be very specific.

If you make a slightest typo and don’t check your manifests during the development process as well as in CI pipelines, your chart’s installation will fail.

Luckily, there are VsCode plugins which help us avoid these mistakes. Yet the tests have to run anyway. You can use either kubeval or kubeconform for that. Kubeconform’s advantage is that it allegedly validates CRDs as well, while kubeval doesn’t.

helm chart unit testing

To unit test helm charts, I’d suggest to use
helm-unittest helm plugin. Using it will ensure that values specified in Kubernetes manifests are as expected. Of course, the owner of the chart has to write the tests themselves. And they are basically yaml files.

For example:

suite: test deployment
templates:
  - deployment.yaml
tests:
  - it: should work
    set:
      image.tag: latest
    asserts:
      - isKind:
          of: Deployment
      - matchRegex:
          path: metadata.name
          pattern: -my-chart$
      - equal:
          path: spec.template.spec.containers[0].image
          value: nginx:latest

helm chart policy validations

These validations will allow to test cross-org policies like: “charts’ images have to come from internal artifact registry” or “charts workloads have not to run as root”

To support policy validations, several tools exist:

Checkov has multiple built-in policies. Both tools support custom written policies.

helm lint

As the docs of this command state, helm linter fails with [ERROR] messages on things that will cause the chart to fail installation.

Test helm charts as part of Release pipeline

Before releasing the chart (i.e. versioning and pushing it to artifact registry), we would like to test that it can be deployed successfully. During Release pipeline after chart’s deployment, we’d like to run Kubernetes integration tests and acceptance tests. Let’s see what I mean by that.

Kubernetes integration tests

Yes, it’s right that Kubernetes ensures the desired state of resources created in Kubernetes cluster. Yet, we still would like to check some things after deployment using kubectl commands. For example, we may want to validate that the number of replicas is correct, volumes were created with nfs mounts, etc… As we are talking about automatic tests only, we need a way to run such Kubernetes validations automatically. In can be done using any official Kubernetes clients, for instance in Python.

Acceptance tests

When I talk about acceptance tests I mean automated tests which treat the app as black box as a customer would do. You may say it’s QA responsibility, yet we are talking here about modern app development which includes automatic testing and not using QA as scapegoat for developers’ sins.

So what do we test during acceptance tests? We’d like to test that the app meets functionality requirements from the customer perspective e.g. that functionality exposed by app public APIs works as expected.

So how to do that in helm context during Release pipeline. In short, we have to run helm test command. This command will run test container which will contain the actual tests run against deployed chart public APIs. We have seen an example of using this command when we deployed helm chart to minikube in docker and tested it. In addition, have a look at the dedicated helm acceptance tests article I wrote.

Summary

That’s it about testing helm charts. As always, feel free to share. If you found this article useful, take a look at the disclaimer for information on how to thank me.

You may also find below articles interesting:

Become a Certified Kubernetes Administrator (CKA)!

Recommended Helm courses on Pluralsight:

Sign up using this link to get exclusive discounts like 50% off your first month or 15% off an annual subscription)

Recommended Kubernetes books on Amazon: