Skip to content

kuadrantctl - CI/CD with Tekton and Argo CD

This guide demonstrates setting up a CI/CD pipeline using Tekton to deploy Kubernetes Gateway API and Kuadrant resources generated by kuadrantctl, from an OpenAPI specification. In this example, these resources are applied directly to the cluster where Tekton is running.

Prerequisites

  • Kuadrant, and all of its pre-requisites, is installed onto a cluster
  • Tekton Pipelines installed on your Kubernetes or OpenShift cluster.
  • kubectl configured to communicate with your cluster (i.e you have a kubectl config available with access to your cluster)
  • Tekton CLI tkn (optional) for easier interaction with Tekton resources.

Setup

First, create a dedicated namespace:

kubectl create namespace petstore

Create a Persistent Volume Claim

For this guide, to store associated Tekton build artifacts, we'll create a PVC in the petstore namespace:

kubectl apply -n petstore -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: tekton-kuadrantctl-pvc
  namespace: petstore
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
EOF

Define the Tekton Task

Define the task that outlines steps to clone a repository, generate Kuadrant and Kubernetes resources using kuadrantctl, and apply them directly to the cluster:

kubectl apply -f - <<'EOF'
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: run-kuadrantctl
  namespace: petstore
spec:
  params:
    - name: gitRepoUrl
      description: URL of the git repository to clone
    - name: gitRevision
      description: Git revision to checkout (branch, tag, sha)
  workspaces:
    - name: source
      description: Workspace to checkout the git repo
    - name: kubeconfig
      description: Workspace containing kubeconfig for Kubernetes cluster access
  steps:
    - name: clean-workspace
      image: alpine:latest
      script: |
        sh -c 'rm -rf $(workspaces.source.path)/* $(workspaces.source.path)/.[!.]* $(workspaces.source.path)/..?*'
    - name: clone
      image: alpine/git:latest
      script: |
        git clone $(params.gitRepoUrl) $(workspaces.source.path)
        cd $(workspaces.source.path)
        git checkout $(params.gitRevision)
    - name: download-kuadrantctl
      image: curlimages/curl:latest
      script: |
        ARCH=$(uname -m)
        case $ARCH in
        x86_64) BIN_ARCH="amd64";;
        arm64) BIN_ARCH="arm64";;
        aarch64) BIN_ARCH="arm64";;
        *) echo "Unsupported architecture: $ARCH" && exit 1 ;;
        esac
        cd $(workspaces.source.path)
        curl -LO "https://github.com/Kuadrant/kuadrantctl/releases/download/v0.2.3/kuadrantctl-v0.2.3-linux-$BIN_ARCH.tar.gz"
        tar -xzf kuadrantctl-v0.2.3-linux-$BIN_ARCH.tar.gz
    - name: run-kuadrantctl
      image: alpine:latest
      script: |
        cd $(workspaces.source.path)
        mkdir -p generated-resources
        ./kuadrantctl generate kuadrant authpolicy --oas openapi.yaml | tee generated-resources/authpolicy.yaml
        ./kuadrantctl generate kuadrant ratelimitpolicy --oas openapi.yaml |  tee generated-resources/ratelimitpolicy.yaml
        ./kuadrantctl generate gatewayapi httproute --oas openapi.yaml | tee generated-resources/httproute.yaml
    - name: apply-resources
      image: bitnami/kubectl
      script: |
        cd $(workspaces.source.path)
        export KUADRANT_ZONE_ROOT_DOMAIN=example.com # domain name used in the HTTPRoute for the petstore sample app
        for file in ./generated-resources/*.yaml; do
          envsubst < "$file" | kubectl apply -n petstore -f - 
        done
EOF

We're using Tekton here with kubectl to apply resources to a cluster. We recommend looking at a tool such as ArgoCD to implement continuous delivery via a GitOps approach. In this scenario, you would:

  • Use kuadrantctl to generate Kubernetes/Kuadrant resources as part a Tekton pipeline
  • Commit these new resources in to a git respository
  • Use ArgoCD to sync these changes via a Git respository to a Kubernetes or OpenShift cluster

Create a Kubeconfig Secret

Note: Important: While this guide uses a kubeconfig secret for simplicity, it is not recommended for production environments. Instead, use a Service Account for enhanced security.

In this guide, we use a kubeconfig secret coupled with role bindings to demonstrate how to provide access for pushing generated resources to a cluster. However, for production setups, employing a Service Account is advised.

To proceed, create a kubeconfig secret in the petstore namespace to provide Tekton with access to your Kubernetes cluster:

kubectl create secret generic kubeconfig-secret --from-file=kubeconfig=/path/to/.kube/config -n petstore

Create an associated ClusterRole and ClusterRoleBinding:

kubectl apply -n petstore -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kuadrant-ci-example-full-access
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kuadrant-ci-example-full-access-binding
subjects:
- kind: ServiceAccount
  name: default
  namespace: petstore
roleRef:
  kind: ClusterRole
  name: kuadrant-ci-example-full-access
  apiGroup: rbac.authorization.k8s.io
EOF

Trigger the TaskRun

Execute the task within the petstore namespace, referencing the kubeconfig secret for cluster access:

In this example, we'll run this task with our Kuadrant Petstore app: https://github.com/kuadrant/api-petstore

kubectl apply -n petstore -f - <<EOF
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: run-kuadrantctl-taskrun
  namespace: petstore
spec:
  taskRef:
    name: run-kuadrantctl
  params:
    - name: gitRepoUrl
      value: "https://github.com/kuadrant/api-petstore.git"
    - name: gitRevision
      value: "main"
  workspaces:
    - name: source
      persistentVolumeClaim:
        claimName: tekton-kuadrantctl-pvc
    - name: kubeconfig
      secret:
        secretName: kubeconfig-secret
EOF

If you have tkn installed, you can easily view the progress of the pipe run:

tkn taskrun list -n petstore
NAME                      STARTED          DURATION   STATUS
run-kuadrantctl-taskrun   12 seconds ago   ---        Running(Pending)
tkn taskrun logs -n petstore -f


[clone] Cloning into '/workspace/source'...
[clone] Already on 'main'
[clone] Your branch is up to date with 'origin/main'.

[download-kuadrantctl]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[download-kuadrantctl]                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 21.4M  100 21.4M    0     0  6601k      0  0:00:03  0:00:03 --:--:-- 8756k

[run-kuadrantctl] {"kind":"AuthPolicy","apiVersion":"kuadrant.io/v1beta2","metadata":{"name":"petstore","namespace":"petstore","creationTimestamp":null,"labels":{"deployment":"petstore","owner":"jbloggs"}},"spec":{"targetRef":{"group":"gateway.networking.k8s.io","kind":"HTTPRoute","name":"petstore","namespace":"petstore"},"routeSelectors":[{"matches":[{"path":{"type":"Exact","value":"/api/v3/store/admin"},"method":"GET"}]}],"rules":{"authentication":{"storeAdmin_api_key":{"credentials":{"customHeader":{"name":"api_key"}},"apiKey":{"selector":{"matchLabels":{"kuadrant.io/apikeys-by":"api_key"}}},"routeSelectors":[{"matches":[{"path":{"type":"Exact","value":"/api/v3/store/admin"},"method":"GET"}]}]}}}},"status":{}}
[run-kuadrantctl] {"kind":"RateLimitPolicy","apiVersion":"kuadrant.io/v1beta2","metadata":{"name":"petstore","namespace":"petstore","creationTimestamp":null,"labels":{"deployment":"petstore","owner":"jbloggs"}},"spec":{"targetRef":{"group":"gateway.networking.k8s.io","kind":"HTTPRoute","name":"petstore","namespace":"petstore"},"limits":{"getInventory":{"routeSelectors":[{"matches":[{"path":{"type":"Exact","value":"/api/v3/store/inventory"},"method":"GET"}]}],"rates":[{"limit":10,"duration":10,"unit":"second"}]},"loginUser":{"routeSelectors":[{"matches":[{"path":{"type":"Exact","value":"/api/v3/user/login"},"method":"GET"}]}],"rates":[{"limit":2,"duration":10,"unit":"second"}]}}},"status":{}}
[run-kuadrantctl] {"kind":"HTTPRoute","apiVersion":"gateway.networking.k8s.io/v1beta1","metadata":{"name":"petstore","namespace":"petstore","creationTimestamp":null,"labels":{"deployment":"petstore","owner":"jbloggs"}},"spec":{"parentRefs":[{"kind":"Gateway","namespace":"kuadrant-multi-cluster-gateways","name":"prod-web"}],"hostnames":["petstore.${KUADRANT_ZONE_ROOT_DOMAIN}"],"rules":[{"matches":[{"path":{"type":"Exact","value":"/api/v3/user/login"},"method":"GET"}],"backendRefs":[{"name":"petstore","namespace":"petstore","port":8080}]},{"matches":[{"path":{"type":"Exact","value":"/api/v3/store/admin"},"method":"GET"}],"backendRefs":[{"name":"petstore","namespace":"petstore","port":8080}]},{"matches":[{"path":{"type":"Exact","value":"/api/v3/store/inventory"},"method":"GET"}],"backendRefs":[{"name":"petstore","namespace":"petstore","port":8080}]}]},"status":{"parents":null}}

[apply-resources] authpolicy.kuadrant.io/petstore created
[apply-resources] httproute.gateway.networking.k8s.io/petstore created
[apply-resources] ratelimitpolicy.kuadrant.io/petstore created

Cleanup

To cleanup:

  • Remove the petstore namespace:
    • kubectl delete ns petstore
  • Remove the ClusterRole and ClusterRoleBinding:
    • kubectl delete clusterrole kuadrant-ci-example-full-access
    • kubectl delete clusterrolebinding kuadrant-ci-example-full-access-binding