Authenticated Rate Limiting with JWTs and Kubernetes RBAC¶
This user guide walks you through an example of how to use Kuadrant to protect an application with policies to enforce:
- authentication based OpenId Connect (OIDC) ID tokens (signed JWTs), issued by a Keycloak server;
- alternative authentication method by Kubernetes Service Account tokens;
- authorization delegated to Kubernetes RBAC system;
- rate limiting by user ID.
In this example, we will protect a sample REST API called Toy Store. In reality, this API is just an echo service that echoes back to the user whatever attributes it gets in the request.
The API listens to requests at the hostnames *.toystore.com, where it exposes the endpoints GET /toy*, POST /admin/toy and DELETE /amind/toy, respectively, to mimic operations of reading, creating, and deleting toy records.
Any authenticated user/service account can send requests to the Toy Store API, by providing either a valid Keycloak-issued access token or Kubernetes token.
Privileges to execute the requested operation (read, create or delete) will be granted according to the following RBAC rules, stored in the Kubernetes authorization system:
| Operation | Endpoint | Required role |
|---|---|---|
| Read | GET /toy* |
toystore-reader |
| Create | POST /admin/toy |
toystore-write |
| Delete | DELETE /admin/toy |
toystore-write |
Each user will be entitled to a maximum of 5rp10s (5 requests every 10 seconds).
Setup the environment¶
Follow this setup doc to set up your environment before continuing with this doc.
Deploy the Toystore example API:¶
API lifecycle¶
Try the API unprotected¶
Export the gateway hostname and port:
export INGRESS_HOST=$(kubectl get gtw kuadrant-ingressgateway -n gateway-system -o jsonpath='{.status.addresses[0].value}')
export INGRESS_PORT=$(kubectl get gtw kuadrant-ingressgateway -n gateway-system -o jsonpath='{.spec.listeners[?(@.name=="http")].port}')
export GATEWAY_URL=$INGRESS_HOST:$INGRESS_PORT
It should return 200 OK.
Note: If the command above fails to hit the Toy Store API on your environment, try forwarding requests to the service and accessing over localhost:
Deploy Keycloak¶
Create the namesapce:
Deploy Keycloak with a bootstrap realm, users, and clients:
kubectl apply -n keycloak -f https://raw.githubusercontent.com/Kuadrant/authorino-examples/main/keycloak/keycloak-deploy.yaml
Note: The Keycloak server may take a couple of minutes to be ready.
Enforce authentication and authorization for the Toy Store API¶
Create a Kuadrant AuthPolicy to configure authentication and authorization:
kubectl apply -f - <<EOF
apiVersion: kuadrant.io/v1
kind: AuthPolicy
metadata:
name: toystore-protection
spec:
targetRef:
group: gateway.networking.k8s.io
kind: HTTPRoute
name: toystore
rules:
authentication:
"keycloak-users":
jwt:
issuerUrl: http://keycloak.keycloak.svc.cluster.local:8080/realms/kuadrant
"k8s-service-accounts":
kubernetesTokenReview:
audiences:
- https://kubernetes.default.svc.cluster.local
overrides:
"sub":
selector: auth.identity.user.username
authorization:
"k8s-rbac":
kubernetesSubjectAccessReview:
user:
selector: auth.identity.sub
response:
success:
filters:
"identity":
json:
properties:
"userid":
selector: auth.identity.sub
EOF
Try the API missing authentication¶
curl -H 'Host: api.toystore.com' http://$GATEWAY_URL/toy -i
# HTTP/1.1 401 Unauthorized
# www-authenticate: Bearer realm="keycloak-users"
# www-authenticate: Bearer realm="k8s-service-accounts"
# x-ext-auth-reason: {"k8s-service-accounts":"credential not found","keycloak-users":"credential not found"}
Try the API without permission¶
Obtain an access token with the Keycloak server:
ACCESS_TOKEN=$(kubectl run token --attach --rm --restart=Never -q --image=curlimages/curl -- http://keycloak.keycloak.svc.cluster.local:8080/realms/kuadrant/protocol/openid-connect/token -s -d 'grant_type=password' -d 'client_id=demo' -d 'username=john' -d 'password=p' -d 'scope=openid' | jq -r .access_token)
Send a request to the API as the Keycloak-authenticated user while still missing permissions:
curl -H "Authorization: Bearer $ACCESS_TOKEN" -H 'Host: api.toystore.com' http://$GATEWAY_URL/toy -i
# HTTP/1.1 403 Forbidden
Create a Kubernetes Service Account to represent a consumer of the API associated with the alternative source of identities k8s-service-accounts:
Obtain an access token for the client-app-1 service account:
Send a request to the API as the service account while still missing permissions:
curl -H "Authorization: Bearer $SA_TOKEN" -H 'Host: api.toystore.com' http://$GATEWAY_URL/toy -i
# HTTP/1.1 403 Forbidden
Grant access to the Toy Store API for user and service account¶
Create the toystore-reader and toystore-writer roles:
kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: toystore-reader
rules:
- nonResourceURLs: ["/toy*"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: toystore-writer
rules:
- nonResourceURLs: ["/admin/toy"]
verbs: ["post", "delete"]
EOF
Add permissions to the user and service account:
| User | Kind | Roles |
|---|---|---|
| john | User registered in Keycloak | toystore-reader, toystore-writer |
| client-app-1 | Kuberentes Service Account | toystore-reader |
kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: toystore-readers
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: toystore-reader
subjects:
- kind: User
name: $(jq -R -r 'split(".") | .[1] | @base64d | fromjson | .sub' <<< "$ACCESS_TOKEN")
- kind: ServiceAccount
name: client-app-1
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: toystore-writers
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: toystore-writer
subjects:
- kind: User
name: $(jq -R -r 'split(".") | .[1] | @base64d | fromjson | .sub' <<< "$ACCESS_TOKEN")
EOF
Q: Can I use Roles and RoleBindings instead of ClusterRoles and ClusterRoleBindings?
Yes, you can.
The example above is for non-resource URL Kubernetes roles. For using Roles and RoleBindings instead of
ClusterRoles and ClusterRoleBindings, thus more flexible resource-based permissions to protect the API,
see the spec for Kubernetes SubjectAccessReview authorization
in the Authorino docs.
Try the API with permission¶
Send requests to the API as the Keycloak-authenticated user:
curl -H "Authorization: Bearer $ACCESS_TOKEN" -H 'Host: api.toystore.com' http://$GATEWAY_URL/toy -i
# HTTP/1.1 200 OK
curl -H "Authorization: Bearer $ACCESS_TOKEN" -H 'Host: api.toystore.com' -X POST http://$GATEWAY_URL/admin/toy -i
# HTTP/1.1 200 OK
Send requests to the API as the Kubernetes service account:
curl -H "Authorization: Bearer $SA_TOKEN" -H 'Host: api.toystore.com' http://$GATEWAY_URL/toy -i
# HTTP/1.1 200 OK
curl -H "Authorization: Bearer $SA_TOKEN" -H 'Host: api.toystore.com' -X POST http://$GATEWAY_URL/admin/toy -i
# HTTP/1.1 403 Forbidden
Enforce rate limiting on requests to the Toy Store API¶
Create a Kuadrant RateLimitPolicy to configure rate limiting:
kubectl apply -f - <<EOF
apiVersion: kuadrant.io/v1
kind: RateLimitPolicy
metadata:
name: toystore
spec:
targetRef:
group: gateway.networking.k8s.io
kind: HTTPRoute
name: toystore
limits:
"per-user":
rates:
- limit: 5
window: 10s
counters:
- expression: auth.identity.userid
EOF
Note: It may take a couple of minutes for the RateLimitPolicy to be applied depending on your cluster.
Try the API rate limited¶
Each user should be entitled to a maximum of 5 requests every 10 seconds.
Note: If the tokens have expired, you may need to refresh them first.
Send requests as the Keycloak-authenticated user:
while :; do curl --write-out '%{http_code}\n' --silent --output /dev/null -H "Authorization: Bearer $ACCESS_TOKEN" -H 'Host: api.toystore.com' http://$GATEWAY_URL/toy | grep -E --color "\b(429)\b|$"; sleep 1; done
Send requests as the Kubernetes service account:
while :; do curl --write-out '%{http_code}\n' --silent --output /dev/null -H "Authorization: Bearer $SA_TOKEN" -H 'Host: api.toystore.com' http://$GATEWAY_URL/toy | grep -E --color "\b(429)\b|$"; sleep 1; done