Adopting Tekton — Cloud Native CI Solution

I have been exploring Tekton as a CI platform recently for my office and off-office work. So I thought to share all my findings with you. Tekton is a powerful yet flexible, Kubernetes-native open-source framework for creating continuous integration and continuous delivery (CI/CD) systems. But before we proceed, let’s see what exactly Cloud Native CI/CD is? What is cloud-native CI/ CD?

Recently we have encountered a DevOps culture shift with cloud-native architectures in software development. Cloud-Native is a new approach to build and run applications. In short, we can say the most popular way to go Cloud-Native consists of Containers + Kubernetes.

With the rise of containerization, both build and deployment stages in CI/CD pipelines continue to be containerized. It has led to the rise and development of various CI and CD tools for the Cloud-Native landscape.

Kubernetes and CI/CD

While running CI/CD pipelines in Kubernetes, some of the challenges we are going to face are:

Images instead of binaries
Clusters: Many environments
Microservices instead of monoliths (Managing dependencies between all services is going to be challenging)
Ephemeral environments.

Hence, we need a solution to these challenges in the form of cloud-native CI/CD tools. Some of the tools listed in the cloud-native landscape are Spinnaker, Argo CD, Tekton, and Jenkins X. Let’s take a deep dive into Tekton now!

Tekton

Tekton provides us with composable, declarative, reproducible, and Cloud-Native tools to easily build the pipelines. Tekton adds a few CRDs and custom resources to your Kubernetes cluster which can be used to make CI/CD pipelines.

Tekton Building Blocks

Step: Step is a container spec (k8s type), the most fundamental building block in Tekton. For instance, running unit tests of application would be a step.

Task: Task is a sequence of steps, runs in sequential order, and most importantly, it runs on the same K8s node, which allows you to have the same environment. For instance, if you mount a volume in a task, it would be shared across each step.

Pipeline: Pipeline is a collection of Tasks that can be run in different ways ( sequentially, concurrently, or a DAG). You can also provide the output of one task to another task even if it runs on a different K8s node.

To invoke these we have :

Instances of Pipeline/Task:

PipelineRun: It is a specific execution of a pipeline that contains runtime data such as parameters and results.
TaskRun: It is a specific execution of a task with runtime data.

PipelineRun and TaskRun bind together your tasks and pipelines with the parameter values and data. They can be triggered manually or on-demand using Tekton triggers.

In this blog, we will see how to trigger a pipeline manually. Later in the next blog, I will give an example to trigger pipeline automatically with webhook configuration. Let’s get started!

How to get started?

Prerequisites :

Kubernetes cluster to test out the changes. Install Kind to create a cluster locally.
Install Tekton on the cluster :

This will Install Tekton in a new namespace called tekton-pipelines :

1kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

To switch to tekton-pipelines namespace where Tekton components are installed use this :

1kubectl config set-context --current --namespace=tekton-pipelines

Install Tekton Dashboard .
After installation, set up port forwarding with Tekton Dashboard:

1kubectl --namespace tekton-pipelines port-forward svc/tekton-dashboard 9097:9097

Set up a container registry. If you do not have one, you can set up a Github container registry by referring to https://nancychauhan.in/github-container-registry

Setting up

Install tasks from Tekton Hub

Install tasks on your Kubernetes namespace. We will use them to build the pipeline later.

git-clone

1kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/git-clone/0.2/git-clone.yaml

kaniko

1kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/kaniko/0.1/kaniko.yaml

Create a secret for the service account

It will provide the credentials for pushing the docker image we build in the pipeline later.

1kubectl create secret docker-registry ghcr --docker-username=$USERNAME --docker-password=$TOKEN --docker-server="https://ghcr.io/v1/"

Apply Tekton manifests

We will set up CI for a sample project: https://github.com/Nancy-Chauhan/keystore

This pipeline will run the tests, build a docker image, and push it to the registry we configured.

Create a service account that allows access to the secrets we just created:

1apiVersion: v1
2kind: ServiceAccount
3metadata:
4  name: tekton-ci
5secrets:
6  - name: ghcr

1kubectl apply -f https://gist.githubusercontent.com/Nancy-Chauhan/a81c3088ff5a446fab026a20fecc20ad/raw/dfd0475c5e1cc0513547f9144e4108b8eeb6f2b0/tekton-ci.yaml

Create a pipeline to run tests and build & push a docker image:

1apiVersion: tekton.dev/v1beta1
2kind: Pipeline
3metadata:
4  name: ci
5spec:
6  params:
7    - name: git-revision
8      type: string
9    - name: git-url
10      type: string
11    - name: image
12      type: string
13  workspaces:
14    - name: source
15  tasks:
16    - name: git-clone
17      taskRef:
18        name: git-clone
19      workspaces:
20        - name: output
21          workspace: source
22      params:
23        - name: revision
24          value: $(params.git-revision)
25        - name: url
26          value: $(params.git-url)
27    - name: build-and-push
28      runAfter: ["git-clone"]
29      taskRef:
30        name: kaniko
31      workspaces:
32        - name: source
33          workspace: source
34      params:
35        - name: IMAGE
36          value: $(params.image)

1kubectl apply -f https://gist.githubusercontent.com/Nancy-Chauhan/d173a53761458b82a858274e8335a163/raw/8219e12ee894ab2c4645f5f593b4641cb8e7e3df/pipeline.yaml

Run the pipeline

Create a PipelineRun to trigger the pipeline, and you can pass information about the build.

Use the below example to create a PipelineRun. Replace the value of the image to refer to your registry and other parameters as necessary.

1apiVersion: tekton.dev/v1beta1
2kind: PipelineRun
3metadata:
4  generateName: keystore-ci-
5spec:
6  pipelineRef:
7    name: ci
8  serviceAccountName: tekton-ci
9  params:
10    - name: git-revision
11      value: master
12    - name: git-url
13      value: https://github.com/Nancy-Chauhan/keystore.git
14    - name: image
15      value: ghcr.io/nancy-chauhan/keystore:latest
16    - name: repo-name
17      value: Nancy-Chauhan/keystore
18    - name: git-commit-sha
19      value: 0d4b1935a9eddd1b3ecc8e921901aa34dfa703e6
20  workspaces:
21    - name: source
22      volumeClaimTemplate:
23        spec:
24          accessModes:
25            - ReadWriteOnce
26          resources:
27            requests:
28              storage: 5Gi

1kubectl create -f keystore-ci.yaml

You can see the status of the pipeline in the Tekton Dashboard:

When the pipeline completes successfully, it posts success status for the commit.

Limitations:

We currently only update the success status as Tekton does not provide completion status to tasks yet. There are workarounds like writing status to a file and using it to send status to Github, but it is a story for another time. If you are interested in tracking the progress, check this:

https://github.com/tektoncd/pipeline/issues/3645

Advantages of using Tekton:

Tekton is cloud-native, runs on an existing Kubernetes cluster.
Tekton provides an out-of-the-box solution for monitoring. Easy to build pipeline using reusable tasks and steps.

Disadvantages :

No out-of-the-box support for authentication.
Pipeline configuration is stored separately from code. To change the pipeline configuration, developers need to apply Kubernetes manifests manually.

Tekton community is quite helpful and quite prompt to reply to all your queries. Check it here: https://github.com/tektoncd/community/blob/master/contact.md

References: https://www.youtube.com/watch?v=sUkvpzr9du8

In the next blog, I will discuss setting up a webhook to automatically trigger the pipeline and monitor Tekton.

Originally Posted at https://medium.com/@_nancychauhan/adopting-tekton-cloud-native-ci-solution-67fb229f4992