kuadrantctl - CI/CD with Tekton and Argo CD¶

This guide demonstrates setting up a CI/CD pipeline by using Tekton to deploy Kubernetes Gateway API and Kuadrant resources generated by kuadrantctl, from an OpenAPI definition. In this example, these resources are applied directly to the cluster where Tekton is running.

Prerequisites¶

Kuadrant, and all of its prerequisites, installed on a Kubernetes or OpenShift cluster.
Tekton Pipelines installed on your cluster.
kubectl configured with access to communicate with your cluster.
Optional: Tekton CLI tkn for easier interaction with Tekton resources.

Procedure¶

Step 1 - Set up your namespace¶

Create a dedicated namespace as follows:

kubectl create namespace petstore

Step 2 - Create a Persistent Volume Claim¶

For this example, to store associated Tekton build artifacts, create a Persistent Volume Claim (PVC) in the petstore namespace as follows:

kubectl apply -n petstore -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: tekton-kuadrantctl-pvc
  namespace: petstore
spec:
  accessModes:

    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
EOF

Step 3 - Define the Tekton Task¶

Define the task that outlines steps to clone a repository, generate Kuadrant and Kubernetes resources by using kuadrantctl, and apply them directly to the cluster as follows:

kubectl apply -f - <<'EOF'
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: run-kuadrantctl
  namespace: petstore
spec:
  params:

    - name: gitRepoUrl
      description: URL of the git repository to clone
    - name: gitRevision
      description: Git revision to checkout (branch, tag, sha)
  workspaces:
    - name: source
      description: Workspace to checkout the git repo
    - name: kubeconfig
      description: Workspace containing kubeconfig for Kubernetes cluster access
  steps:
    - name: clean-workspace
      image: alpine:latest
      script: |
        sh -c 'rm -rf $(workspaces.source.path)/* $(workspaces.source.path)/.[!.]* $(workspaces.source.path)/..?*'
    - name: clone
      image: alpine/git:latest
      script: |
        git clone $(params.gitRepoUrl) $(workspaces.source.path)
        cd $(workspaces.source.path)
        git checkout $(params.gitRevision)
    - name: download-kuadrantctl
      image: curlimages/curl:latest
      script: |
        ARCH=$(uname -m)
        case $ARCH in
        x86_64) BIN_ARCH="amd64";;
        arm64) BIN_ARCH="arm64";;
        aarch64) BIN_ARCH="arm64";;
        *) echo "Unsupported architecture: $ARCH" && exit 1 ;;
        esac
        cd $(workspaces.source.path)
        curl -LO "https://github.com/Kuadrant/kuadrantctl/releases/download/v0.2.3/kuadrantctl-v0.2.3-linux-$BIN_ARCH.tar.gz"
        tar -xzf kuadrantctl-v0.2.3-linux-$BIN_ARCH.tar.gz
    - name: run-kuadrantctl
      image: alpine:latest
      script: |
        cd $(workspaces.source.path)
        mkdir -p generated-resources
        ./kuadrantctl generate kuadrant authpolicy --oas openapi.yaml | tee generated-resources/authpolicy.yaml
        ./kuadrantctl generate kuadrant ratelimitpolicy --oas openapi.yaml |  tee generated-resources/ratelimitpolicy.yaml
        ./kuadrantctl generate gatewayapi httproute --oas openapi.yaml | tee generated-resources/httproute.yaml
    - name: apply-resources
      image: bitnami/kubectl
      script: |
        cd $(workspaces.source.path)
        export KUADRANT_ZONE_ROOT_DOMAIN=example.com # domain name used in the HTTPRoute for the petstore sample app
        for file in ./generated-resources/*.yaml; do
          envsubst < "$file" | kubectl apply -n petstore -f - 
        done
EOF

Note: This example uses Tekton with kubectl to apply resources to a cluster. It is best to use a tool such as Argo CD to implement continuous delivery by using a GitOps approach. In this scenario, you would do the following:

Use kuadrantctl to generate Kubernetes and Kuadrant resources as part a Tekton pipeline.
Commit these new resources to a Git repository.
Use ArgoCD to sync these changes from the Git repository to a Kubernetes or OpenShift cluster.

Step 4 - Create a Kubeconfig secret¶

Important: While this guide uses a kubeconfig secret for simplicity, do not use this in production environments. Instead, use a service account for enhanced security.

This example uses a kubeconfig secret and role bindings to demonstrate how to provide access for pushing generated resources to a cluster. However, for production setups, employing a service account is best.

To proceed, create a kubeconfig secret in the petstore namespace to provide Tekton with access to your Kubernetes cluster as follows:

kubectl create secret generic kubeconfig-secret --from-file=kubeconfig=/path/to/.kube/config -n petstore

Create an associated ClusterRole and ClusterRoleBinding as follows:

kubectl apply -n petstore -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kuadrant-ci-example-full-access
rules:

- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kuadrant-ci-example-full-access-binding
subjects:
- kind: ServiceAccount
  name: default
  namespace: petstore
roleRef:
  kind: ClusterRole
  name: kuadrant-ci-example-full-access
  apiGroup: rbac.authorization.k8s.io
EOF

Step 5 - Trigger the TaskRun¶

Execute the task from the petstore namespace, referencing the kubeconfig secret for cluster access as follows:

This example runs this task with the Kuadrant Petstore app: https://github.com/kuadrant/api-petstore.

kubectl apply -n petstore -f - <<EOF
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: run-kuadrantctl-taskrun
  namespace: petstore
spec:
  taskRef:
    name: run-kuadrantctl
  params:

    - name: gitRepoUrl
      value: "https://github.com/kuadrant/api-petstore.git"
    - name: gitRevision
      value: "main"
  workspaces:
    - name: source
      persistentVolumeClaim:
        claimName: tekton-kuadrantctl-pvc
    - name: kubeconfig
      secret:
        secretName: kubeconfig-secret
EOF

If you have tkn installed, you can easily view the progress of the pipe run as follows:

tkn taskrun list -n petstore
NAME                      STARTED          DURATION   STATUS
run-kuadrantctl-taskrun   12 seconds ago   ---        Running(Pending)

tkn taskrun logs -n petstore -f


[clone] Cloning into '/workspace/source'...
[clone] Already on 'main'
[clone] Your branch is up to date with 'origin/main'.

[download-kuadrantctl]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[download-kuadrantctl]                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 21.4M  100 21.4M    0     0  6601k      0  0:00:03  0:00:03 --:--:-- 8756k

[run-kuadrantctl] {"kind":"AuthPolicy","apiVersion":"kuadrant.io/v1beta2","metadata":{"name":"petstore","namespace":"petstore","creationTimestamp":null,"labels":{"deployment":"petstore","owner":"jbloggs"}},"spec":{"targetRef":{"group":"gateway.networking.k8s.io","kind":"HTTPRoute","name":"petstore","namespace":"petstore"},"routeSelectors":[{"matches":[{"path":{"type":"Exact","value":"/api/v3/store/admin"},"method":"GET"}]}],"rules":{"authentication":{"storeAdmin_api_key":{"credentials":{"customHeader":{"name":"api_key"}},"apiKey":{"selector":{"matchLabels":{"kuadrant.io/apikeys-by":"api_key"}}},"routeSelectors":[{"matches":[{"path":{"type":"Exact","value":"/api/v3/store/admin"},"method":"GET"}]}]}}}},"status":{}}
[run-kuadrantctl] {"kind":"RateLimitPolicy","apiVersion":"kuadrant.io/v1beta2","metadata":{"name":"petstore","namespace":"petstore","creationTimestamp":null,"labels":{"deployment":"petstore","owner":"jbloggs"}},"spec":{"targetRef":{"group":"gateway.networking.k8s.io","kind":"HTTPRoute","name":"petstore","namespace":"petstore"},"limits":{"getInventory":{"routeSelectors":[{"matches":[{"path":{"type":"Exact","value":"/api/v3/store/inventory"},"method":"GET"}]}],"rates":[{"limit":10,"duration":10,"unit":"second"}]},"loginUser":{"routeSelectors":[{"matches":[{"path":{"type":"Exact","value":"/api/v3/user/login"},"method":"GET"}]}],"rates":[{"limit":2,"duration":10,"unit":"second"}]}}},"status":{}}
[run-kuadrantctl] {"kind":"HTTPRoute","apiVersion":"gateway.networking.k8s.io/v1beta1","metadata":{"name":"petstore","namespace":"petstore","creationTimestamp":null,"labels":{"deployment":"petstore","owner":"jbloggs"}},"spec":{"parentRefs":[{"kind":"Gateway","namespace":"kuadrant-multi-cluster-gateways","name":"prod-web"}],"hostnames":["petstore.${KUADRANT_ZONE_ROOT_DOMAIN}"],"rules":[{"matches":[{"path":{"type":"Exact","value":"/api/v3/user/login"},"method":"GET"}],"backendRefs":[{"name":"petstore","namespace":"petstore","port":8080}]},{"matches":[{"path":{"type":"Exact","value":"/api/v3/store/admin"},"method":"GET"}],"backendRefs":[{"name":"petstore","namespace":"petstore","port":8080}]},{"matches":[{"path":{"type":"Exact","value":"/api/v3/store/inventory"},"method":"GET"}],"backendRefs":[{"name":"petstore","namespace":"petstore","port":8080}]}]},"status":{"parents":null}}

[apply-resources] authpolicy.kuadrant.io/petstore created
[apply-resources] httproute.gateway.networking.k8s.io/petstore created
[apply-resources] ratelimitpolicy.kuadrant.io/petstore created

Step 6 - Cleanup¶

Clean up your resources as follows:

Remove the petstore namespace:
kubectl delete ns petstore
Remove the ClusterRole and ClusterRoleBinding:
kubectl delete clusterrole kuadrant-ci-example-full-access
kubectl delete clusterrolebinding kuadrant-ci-example-full-access-binding