Categories
CI/CD DevOps

Terraform Pipelines on GitLab

Today, I’ll show how to run a sample Terraform pipeline on GitLab. The pipeline will include GitLab Terraform CI templates. If you later find this article useful take a look at the disclaimer for information on how to thank me.

Table Of Contents
  1. Terraform sample GitLab project
  2. Terraform GitLab CI templates
  3. Terraform on GitLab Demo
  4. Summary

Terraform sample GitLab project

I’ll use Terraform to provision Kubernetes cluster on Linode with all the necessary resources like VMs and networking.

The cluster won’t be a managed Kubernetes cluster we already created using CLI and UI, but rather a full blown cluster you manage with master and secondary Kubernetes nodes. Have a look at the Terraform code if you want to dive deeper into provisioning of Kubernetes clusters.

Terraform GitLab CI templates

Have a look at .gitlab-ci.yml in the project. It imports Terraform.gitlab-ci.yml – one of GitLab Terraform templates. Using them makes it very easy to head start validating, testing, building and deploying a Terraform project.

include:
 - template: Terraform.gitlab-ci.yml

terraform-build:
 extends: build
 script:
    - cd "${TF_ROOT}"
    - terraform plan -out=plan.cache -var linode_token=$LINODE_CLI_TOKEN
  

terraform-deploy:
  extends: deploy
  script:
    - cd "${TF_ROOT}"
    - terraform apply "plan.cache"

Terraform.gitlab-ci.yml is a stable template which contains implementation out of the box for below GitLab pipeline stages:

include:
  - template: Terraform/Base.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml
  - template: Jobs/SAST-IaC.gitlab-ci.yml   # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml

stages:
  - validate
  - test
  - build
  - deploy
  - cleanup

fmt:
  extends: .terraform:fmt
  needs: []

validate:
  extends: .terraform:validate
  needs: []

build:
  extends: .terraform:build

deploy:
  extends: .terraform:deploy
  dependencies:
    - build
  environment:
    name: $TF_STATE_NAME

Let’s go over each stage.

Validate

Validate stage has 2 jobs:

  • .terraform:fmt
  • .terraform:validate

They run gitlab-terraform fmt and gitlab-terraform validate respectively which correspond to terraform fmt and validate actions. gitlab-terraform is GitLab wrapper of a terraform binary.

As GitLab docs put it:

This script is a thin wrapper around the terraform binary. Its main purpose is to serve the Infrastructure as code with Terraform and GitLab , by extracting some of the standard configuration a user would need to set up to use the Terraform backend on GitLab as well as the Terraform merge request widget.

Test

Test phase runs kics-iac-sast job which runs SAST (Static Application Security Testing). The job checks the code for known vulnerabilities using kics analyzer.

Build

Build job runs gitlab-terraform plan which is basically terraform plan. Note that I’ve overridden build job with terraform-build in the project’s .gitlab-ci.yml in order to run terraform plan -out=cache.plan -var linode_token=$LINODE_CLI_TOKEN.

Deploy

Now, deploy job is a manual GitLab job (you have to invoke it manually via GitLab UI) which runs gitlab-terraform apply which is basically terraform apply. I’ve also overridden it with terraform apply "cache.plan".

Terraform on GitLab Demo

Let’s now see a demo for using Terraform code to create a Kubernetes cluster on Linode.

Demo Prerequisites

You’ll need your own Linode account. Create one and get 100$ credit using this link. Then, generate account’s access token with write permissions to Kubernetes scope. Finally, add the token as CI/CD masked variable named LINODE_CLI_TOKEN in the project settings.

CI Pipeline

CI pipeline will run on each new merge request to main branch or an update of the existing one. Let’s do a sample change, commit and push it in the feature branch, open the merge request and see the merge request pipeline in action.

As we can see, all mentioned above jobs ran except deploy. You’ll have to run it manually on GitLab UI after merging the merge request to main branch.

CD Pipeline

Let’s now merge the merge request to main branch. All the jobs will run again and deploy job will require manual invocation.

Let’s run it. Deploy will provision Kubernetes cluster on Linode after several minutes. After successful provisioning you should see worker and master nodes IPs and kubeconfig file path kubectl_config = "default.conf".

Run kubectl command against the cluster to check its status:

kubectl --kubeconfig default.conf get nodes
NAME               STATUS     ROLES    AGE     VERSION
default-master-1   NotReady   master   5m57s   v1.18.13
default-node-1     NotReady   <none>   5m31s   v1.18.13
default-node-2     NotReady   <none>   5m31s   v1.18.13
default-node-3     NotReady   <none>   5m17s   v1.18.13

You see that the cluster was provisioned, but its nodes are in NotReady status. You may want to investigate why. However, I focused more on Terraform pipelines on GitLab in this article.

Summary

That’s it about Terraform pipelines on GitLab. As always, feel free to share.

If you found this article useful, take a look at the disclaimer for information on how to thank me.

You may also find below articles interesting:

Bonus:

Recommended Kubernetes courses on Pluralsight:

Sign up using this link to get exclusive discounts like 50% off your first month or 15% off an annual subscription)

Recommended Terraform books on Amazon.