Install Kuadrant on an OpenShift cluster¶
Note
You must perform these steps on each OpenShift cluster that you want to use Kuadrant on.
In this document we use AWS route 53 as the example setup.
Warning
Kuadrant uses a number of labels to search and filter resources on the cluster.
All required labels are formatted as kuadrant.io/*.
Removal of any labels with the prefix may cause unexpected behaviour and degradation of the product.
Prerequisites¶
- OpenShift Container Platform 4.16.x or later with community Operator catalog available.
- AWS/Azure or GCP with DNS capabilities.
- Accessible Redis instance.
Procedure¶
Step 1 - Set up your environment¶
We use env vars for convenience only here. If you know these values you can setup the required yaml files in anyway that suites your needs.
export AWS_ACCESS_KEY_ID=xxxxxxx # Key ID from AWS with Route 53 access
export AWS_SECRET_ACCESS_KEY=xxxxxxx # Access key from AWS with Route 53 access
export REDIS_URL=redis://user:xxxxxx@some-redis.com:10340 # A Redis cluster URL
Step 2 - Install Gateway API v1¶
Before you can use Kuadrant, you must install Gateway API v1 as follows:
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml
Step 3 - Install cert-manager¶
Before you can use Kuadrant, you must install cert-manager. Cert-Manager is used by kuadrant to manage TLS certificates for your gateways.
The minimum supported version of cert-manager is v1.14.0.
Install one of the different flavours of the Cert-Manager.
Install community version of the cert-manager¶
Consider installing cert-manager via OperatorHub, which you can do from the OpenShift web console.
More installation options at cert-manager.io
Install cert-manager Operator for Red Hat OpenShift¶
You can install the cert-manager Operator for Red Hat OpenShift by using the web console.
Note: Before using Kuadrant's
TLSPolicyyou will need to setup a certificate issuer refer to the cert-manager docs for more details
Step 4 - (Optional) Install and configure Istio with the Sail Operator¶
Note
Skip this step if planing to use Envoy Gateway as Gateway API provider
Kuadrant integrates with Istio as a Gateway API provider. You can set up an Istio-based Gateway API provider by using the Sail Operator.
Install Istio¶
To install the Istio Gateway provider, run the following commands:
kubectl apply -f - <<EOF
kind: OperatorGroup
apiVersion: operators.coreos.com/v1
metadata:
name: sail
namespace: gateway-system
spec:
upgradeStrategy: Default
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: sailoperator
namespace: gateway-system
spec:
channel: 3.0-dp1
installPlanApproval: Automatic
name: sailoperator
source: community-operators
sourceNamespace: openshift-marketplace
EOF
Check the status of the installation as follows:
When ready, the status will change from installing to complete.
Configure Istio¶
To configure the Istio Gateway API provider, run the following command:
kubectl apply -f - <<EOF
apiVersion: operator.istio.io/v1alpha1
kind: Istio
metadata:
name: default
spec:
version: v1.21.0
namespace: gateway-system
# Disable autoscaling to reduce dev resources
values:
pilot:
autoscaleEnabled: false
EOF
Wait for Istio to be ready as follows:
Step 5 - (Optional) Install Envoy Gateway as a Gateway API provider¶
Note
Skip this step if planing to use Istio as Gateway API provider
Note
There are several ways to install Envoy Gateway (via egctl, Helm chart or Kubernetes yaml) - this is just an example for starting from a bare Kubernetes cluster.
helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.1.0 -n envoy-gateway-system --create-namespace
Enable EnvoyPatchPolicy feature:
TMP=$(mktemp -d)
kubectl get configmap -n envoy-gateway-system envoy-gateway-config -o jsonpath='{.data.envoy-gateway\.yaml}' > ${TMP}/envoy-gateway.yaml
yq e '.extensionApis.enableEnvoyPatchPolicy = true' -i ${TMP}/envoy-gateway.yaml
kubectl create configmap -n envoy-gateway-system envoy-gateway-config --from-file=envoy-gateway.yaml=${TMP}/envoy-gateway.yaml -o yaml --dry-run=client | kubectl replace -f -
kubectl rollout restart deployment envoy-gateway -n envoy-gateway-system
Wait for Envoy Gateway to become available::
kubectl wait --timeout=5m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available
Step 6 - Optional: Configure observability and metrics¶
Kuadrant provides a set of example dashboards that use known metrics exported by Kuadrant and Gateway components to provide insight into different components of your APIs and Gateways. While not essential, it is best to set up an OpenShift monitoring stack. This section provides links to OpenShift and Thanos documentation on configuring monitoring and metrics storage.
You can set up user-facing monitoring by following the steps in the OpenShift documentation on configuring the monitoring stack.
If you have user workload monitoring enabled, it is best to configure remote writes to a central storage system such as Thanos:
The example dashboards and alerts for observing Kuadrant functionality use low-level CPU metrics and network metrics available from the user monitoring stack in OpenShift. They also use resource state metrics from Gateway API and Kuadrant resources.
To scrape these additional metrics, you can install a kube-state-metrics instance, with a custom resource configuration as follows:
kubectl apply -f https://raw.githubusercontent.com/Kuadrant/kuadrant-operator/main/config/observability/openshift/kube-state-metrics.yaml
kubectl apply -k https://github.com/Kuadrant/gateway-api-state-metrics?ref=main
To enable request metrics in Istio, you must create a telemetry resource as follows:
kubectl apply -f https://raw.githubusercontent.com/Kuadrant/kuadrant-operator/main/config/observability/openshift/telemetry.yaml
If you have Grafana installed in your cluster, you can import the example dashboards and alerts.
For example installation details, see installing Grafana on OpenShift. When installed, you must add your Thanos instance as a data source to Grafana. Alternatively, if you are using only the user workload monitoring stack in your OpenShift cluster, and not writing metrics to an external Thanos instance, you can set up a data source to the thanos-querier route in the OpenShift cluster.
Step 7 - Setup the catalogsource¶
Before installing the Kuadrant Operator, you must enter the following commands to set up secrets that you will use later:
Set up a CatalogSource as follows:
kubectl apply -f - <<EOF
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: kuadrant-operator-catalog
namespace: kuadrant-system
spec:
sourceType: grpc
image: quay.io/kuadrant/kuadrant-operator-catalog:v0.11.0
displayName: Kuadrant Operators
publisher: grpc
updateStrategy:
registryPoll:
interval: 45m
EOF
Step 8 - Install the Kuadrant Operator¶
To install the Kuadrant Operator, enter the following command:
kubectl apply -f - <<EOF
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: kuadrant-operator
namespace: kuadrant-system
spec:
channel: stable
installPlanApproval: Automatic
name: kuadrant-operator
source: kuadrant-operator-catalog
sourceNamespace: kuadrant-system
---
kind: OperatorGroup
apiVersion: operators.coreos.com/v1
metadata:
name: kuadrant
namespace: kuadrant-system
spec:
upgradeStrategy: Default
EOF
Wait for the Kuadrant Operators to be installed as follows:
After some time, this command should return complete.
Set up a DNSProvider¶
The example here is for AWS Route 53. It is important the secret for the DNSProvider is setup in the same namespace as the gateway.
kubectl -n ingress-gateway create secret generic aws-credentials \
--type=kuadrant.io/aws \
--from-literal=AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
--from-literal=AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
For more details on other providers take a look at DNS Providers
Step 9 - Install Kuadrant Components¶
To trigger your Kuadrant deployment, enter the following command:
kubectl apply -f - <<EOF
apiVersion: kuadrant.io/v1beta1
kind: Kuadrant
metadata:
name: kuadrant
namespace: kuadrant-system
EOF
Wait for Kuadrant to be ready as follows:
This will setup and configure a number of Kuadrant subcomponents. Some of these can also take additional configuration:
- Authorino (Enforcement Component for AuthPolicy)
- Learn More: (Authorino CRD)[https://docs.kuadrant.io/latest/authorino-operator/#the-authorino-custom-resource-definition-crd]
- Limitador (Enforcement Component for RateLimitPolicy)
- Learn More:(Limitador CRD)[https://docs.kuadrant.io/latest/limitador-operator/#features]
- DNS Operator (Enforcement Component for DNSPOlicy)
Configuring Redis Storage for Limitador¶
Redis credentials for storage of rate limiting counters¶
In this installation we will show how to configure ratelimiting counters to be stored in redis. Before we go further we need to setup a redis secret to use later:
Update limitador config¶
To configure redis storage for Limatador, we must update the Limitador custom resource to use the secret we created:
You can run a command like the one below to add this configuration:
kubectl patch limitador limitador --type=merge -n kuadrant-system -p '
spec:
storage:
redis:
configSecretRef:
name: redis-config
'
Check that limitador is back to ready:
Kuadrant is now ready to use.
Step 10 - Configure the Kuadrant Console Plugin¶
When running on OpenShift, the Kuadrant Operator will automatically install and configure the Kuadrant dynamic console plugin.
Enable the Console Plugin¶
To enable the Kuadrant console plugin:
- Log in to OpenShift or OKD as an administrator.
- Switch to the Admin perspective.
- Navigate to Home > Overview.
- In the Dynamic Plugins section of the status box, click View all.
- In the Console plugins area, find the
kuadrant-consoleplugin. It should be listed but disabled. - Click the Disabled button next to the
kuadrant-consoleplugin. - Select the Enabled radio button, and then click Save.
- Wait for the plugin status to change to Loaded.
Once the plugin is loaded, refresh the console. You should see a new Kuadrant section in the navigation sidebar.