The TokenRateLimitPolicy Custom Resource Definition (CRD)¶

TokenRateLimitPolicy¶

Field	Type	Required	Description
`spec`	TokenRateLimitPolicySpec	Yes	The specification for TokenRateLimitPolicy custom resource
`status`	TokenRateLimitPolicyStatus	No	The status for the custom resource

TokenRateLimitPolicySpec¶

Field	Type	Required	Description
`targetRef`	LocalPolicyTargetReferenceWithSectionName	Yes	Reference to a Kubernetes resource that the policy attaches to. For more info
`defaults`	MergeableTokenRateLimitPolicySpec	No	Default limit definitions. This field is mutually exclusive with the `limits` field
`overrides`	MergeableTokenRateLimitPolicySpec	No	Overrides limit definitions. This field is mutually exclusive with the `limits` field and `defaults` field. This field is only allowed for policies targeting `Gateway` in `targetRef.kind`
`limits`	MapTokenLimit>	No	Limit definitions. This field is mutually exclusive with the `defaults` field

LocalPolicyTargetReferenceWithSectionName¶

Field	Type	Required	Description
`LocalPolicyTargetReference`	LocalPolicyTargetReference	Yes	Reference to a local policy target.
`sectionName`	SectionName	No	Section name for further specificity (if needed).

LocalPolicyTargetReference¶

Field	Type	Required	Description
`group`	`Group`	Yes	Group of the target resource.
`kind`	`Kind`	Yes	Kind of the target resource.
`name`	`ObjectName`	Yes	Name of the target resource.

SectionName¶

Field	Type	Required	Description
SectionName	v1.SectionName (String)	Yes	SectionName is the name of a section in a Kubernetes resource. In the following resources, SectionName is interpreted as the following: Gateway: Listener name HTTPRoute: HTTPRouteRule name * Service: Port name

MergeableTokenRateLimitPolicySpec¶

Field	Type	Required	Description
`strategy`	String	No	Merge strategy to apply when merging with other policies. Values: `atomic` (default), `merge`
`limits`	MapTokenLimit>	Yes	Map of named token-based rate limit configurations

TokenLimit¶

Field	Type	Required	Description
`rates`	[]Rate	No	List of rate limit details including limit and window. If not specified, no rate limits are applied for this limit definition
`when`	[]WhenPredicate	No	List of predicates for this limit. Used in combination with top-level predicates
`counters`	[]Counter	No	CEL expressions that define counter keys for rate limiting. If not specified, rate limiting will be applied globally without user-specific tracking
`reservation`	Reservation	No	Tunes token reservation for this limit. Only takes effect when the Kuadrant CR `spec.tokenRateLimiting.mode` is `Reservation` (the default). Ignored in `Optimistic` mode.

Reservation¶

Configures how many tokens are reserved on request arrival and for how long, when the cluster is in Reservation mode (see the Kuadrant CR tokenRateLimiting reference). Both fields are optional; when the whole reservation block is omitted, defaults are generated.

Field	Type	Required	Description
`amount`	Integer or String	No	Either a literal integer number of tokens, or a CEL expression evaluating to the number of tokens (`uint`), to reserve on request arrival. Defaults to `0` when omitted.
`ttl`	String	No	CEL expression evaluating to the maximum duration (`google.protobuf.Duration`) the reservation is held before it expires. When omitted, the value falls back to the route's `HTTPRoute.spec.rules[].timeouts.backendRequest`; if that is also unset, `ttl` is left unset and Limitador applies its own default.

amount defaults to 0, which reserves no capacity at all. 0 is a documented Limitador short-circuit: the Reserve/Commit calls still happen, but no capacity is held, so Reservation mode behaves identically to Optimistic mode for any limit that doesn't set amount explicitly — no protection against the concurrent-request race RFC 0021 exists to close. This is intentional: Reservation is the cluster-wide default mode, so a 0 default keeps upgrading behavior-neutral for every TokenRateLimitPolicy that predates reservations, instead of silently reserving an arbitrary flat amount for policies that were never tuned for it. To get real protection against concurrent-request races, amount must be set explicitly to a meaningful, non-zero estimate of tokens consumed per request.

The reserved amount is an estimate: once the upstream responds, the actual usage.total_tokens is committed and the unused portion of the reservation is released.

When omitted, ttl defaults to the route's own HTTPRoute.spec.rules[].timeouts.backendRequest, since a reservation only needs to survive as long as the request it protects can legitimately run; setting it much larger than that only extends how long an abandoned reservation (e.g. a disconnected client) blocks capacity for no benefit.

Example — reserve a flat 8000 tokens per request and hold the reservation for 30s:

limits:
  chat:
    rates:

    - limit: 100000
      window: 1h
    reservation:
      amount: 8000
      ttl: 'duration("30s")'

amount also accepts a CEL expression as a quoted string, e.g. amount: "1 + 1". Note: expressions that read the request body (e.g. a requestBodyJSON(...)-based token estimate) are not yet supported for reservation.amount.

Rate¶

Field	Type	Required	Description
`limit`	Number	Yes	Maximum token count allowed for the given window
`window`	Duration	Yes	Time window for the limit (e.g., "1h", "24h", "1m", "1d")

WhenPredicate¶

Field	Type	Required	Description
`predicate`	String	Yes	CEL expression that must evaluate to true for the limit to apply. See Well-known Attributes

Counter¶

Field	Type	Required	Description
`expression`	String	Yes	CEL expression that defines the counter key for rate limiting. See Well-known Attributes

TokenRateLimitPolicyStatus¶

The status object for TokenRateLimitPolicy follows the PolicyStatus pattern from Gateway API.

Field	Type	Description
`observedGeneration`	Number	Generation of the resource that was last reconciled
`conditions`	[]Condition	Current state of the policy

Condition¶

Standard Kubernetes condition fields following Gateway API conventions:

Field	Type	Description
`type`	String	Type of condition (e.g., "Accepted", "Enforced")
`status`	String	Status of the condition ("True", "False", "Unknown")
`observedGeneration`	Number	Generation observed when this condition was last updated
`lastTransitionTime`	Timestamp	Last time the condition transitioned from one status to another
`reason`	String	Machine-readable reason for the condition's last transition
`message`	String	Human-readable message indicating details about the last transition

Token Usage Tracking¶

TokenRateLimitPolicy automatically tracks token consumption from AI/LLM responses by monitoring the usage.total_tokens field in response bodies. This enables accurate usage-based rate limiting where:

Request Phase: The policy evaluates predicates and descriptors during the request
Response Phase: The policy extracts actual token usage from the response body
Rate Limiting: Limitador receives the actual token count as hits_addend for precise accounting

Supported Response Format¶

The policy automatically parses token usage from response bodies in the following format:

{
  "usage": {
    "total_tokens": 150,
    "prompt_tokens": 100,
    "completion_tokens": 50
  }
}

This is compatible with OpenAI-style API responses and similar AI/LLM services.

What's actually checked: Token extraction looks for a single JSON pointer, /usage/total_tokens, in the response body. Any backend that returns that exact field works out of the box, including OpenAI Chat Completions (/v1/chat/completions), OpenAI legacy Completions (/v1/completions), OpenAI Embeddings, and OpenAI-compatible backends like vLLM, kServe, Ollama, Azure OpenAI, and Gemini's OpenAI-compatibility endpoint.

Not currently supported: Providers that return token usage under a different field name — Anthropic's /v1/messages (usage.input_tokens / usage.output_tokens, no total_tokens) or Google Gemini's native endpoint (usageMetadata.totalTokenCount) — are not parsed. Multi-provider support is tracked in #1864.

Streaming Support: Both streaming and non-streaming responses are supported:

Non-streaming: Works with stream: false or when stream is omitted
Streaming: Requires "stream": true and "stream_options": { "include_usage": true } to extract usage from the final stream event

CEL Expression Context¶

TokenRateLimitPolicy provides access to request attributes through CEL expressions. For a comprehensive list of available attributes, see the Well-known Attributes RFC.

Common attributes include:

Context	Available Attributes	Example Usage
Request	`request.method`, `request.url_path`, `request.headers`	`request.method == "POST"`
Authentication	`auth.identity.`, `request.auth.claims.`	`auth.identity.userid`, `request.auth.claims["tier"]`
Remote Address	`source.address`, `source.port`	`source.address`

Examples¶

Basic Token Rate Limiting¶

apiVersion: kuadrant.io/v1alpha1
kind: TokenRateLimitPolicy
metadata:
  name: basic-token-limit
  namespace: gateway-system
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: ai-gateway
  limits:
    global:
      rates:

      - limit: 100000
        window: 1h

User-Based Token Limiting¶

apiVersion: kuadrant.io/v1alpha1
kind: TokenRateLimitPolicy
metadata:
  name: user-token-limits
  namespace: gateway-system
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: api-gateway
  limits:
    free:
      rates:

      - limit: 50000
        window: 24h
      when:
      - predicate: request.path == "/v1/chat/completions"
      - predicate: 'auth.identity.groups.split(",").exists(g, g == "free")'
      counters:
      - expression: auth.identity.userid
    gold:
      rates:
      - limit: 200000
        window: 24h
      when:
      - predicate: request.path == "/v1/chat/completions"
      - predicate: 'auth.identity.groups.split(",").exists(g, g == "gold")'
      counters:
      - expression: auth.identity.userid

Gateway Overrides¶

apiVersion: kuadrant.io/v1alpha1
kind: TokenRateLimitPolicy
metadata:
  name: org-wide-limits
  namespace: gateway-system
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: api-gateway
  overrides:
    strategy: atomic
    limits:
      org-quota:
        rates:

        - limit: 1000000
          window: 24h
        counters:
        - expression: auth.identity.org_id