Anonymous authority
A shared provider key becomes the effective identity. The log shows an application called a model—not which person, workload, or delegated agent exercised whose authority.
EmpowerID Identity Fabric · LLM Gateway
Govern model access before inference
Apply identity, delegated authority, prompt intent, model policy, and current spend state before a governed request reaches an LLM provider—then create signed evidence for completed allowed calls.
Fabric catalog name EmpowerID LLM Gateway Authorization is not a wrapper around inference. It is the decision that determines whether inference should occur.
OpenAI- and Anthropic-compatible routes · Multi-provider control · AuthZEN-compatible policy
A provider key is not an identity
Copilots, workflows, and agents now choose models, consume data, and act on behalf of human and machine subjects. Possession of a provider key proves only that a client can reach a provider—not whose business authority is being exercised, or whether that authority permits the current intent.
Policy on paper
Acceptable-use rules, model approval lists, and data-handling requirements live in documents and ticket workflows while model requests execute continuously, unexamined at runtime.
Spend reported too late
One person can start an agent loop that makes many invocations; a model choice can change cost by an order of magnitude. The invoice reports it after the money is gone.
May this subject, acting under this authority, use this model for this apparent purpose under current policy and spend conditions — and what verifiable evidence links the decision to the model use that followed?
Classify · Authorize · Budget · Prove
Classify
Inline prompt classification exports high-confidence intent and data labels to policy—analytics topics stay separate so exploratory taxonomy never becomes an accidental deny engine.
Authorize
AuthZEN-compatible PDP evaluation with subject, delegation status, model, intent labels, estimated cost, and spend state—every candidate model re-authorized, not config-substituted.
Budget
Estimated cost and authoritative consumed spend evaluated before the provider call—deny, clamp tokens, or route to a lower-cost permitted model before charges are incurred.
Prove
Signed, hash-linked receipts for completed allowed calls bind policy context to measured usage and the effective model actually used—not just the one requested.
Traffic policy versus governed authorization
Multi-provider access, credentials, metrics, retries, rate limits, and operational logs are necessary baseline capabilities. Production systems also need to answer whether a subject may perform an action on a resource under current context.
| Traffic-oriented control | Governed authorization control |
|---|---|
| API key or gateway credential | Human, application, workload, or agent identity — with delegation |
| Route and provider | Model resource and allowed use for this subject and intent |
| Request volume and rate limits | Subject-specific spend state, evaluated before the provider call |
| Static prompt filter | High-confidence intent and data labels supplied to policy |
| Configured fallback model | Model switching in which every candidate is re-authorized by policy |
| Local gateway configuration | Shared policy decision across enforcement points |
| Operational log | Decision-linked, signed evidence for completed allowed use |
The governed model-invocation lifecycle
-
Step 1
Establish the subject
Authenticate through an Identity Fabric session or personal access token; resolve a stable subject and, for agents, the identity chain. A token that names an agent is not enough if delegated authority has expired or does not cover the requested use.
-
Step 2
Classify the request
Inline prompt classification derives intent and data context. Only allowlisted, high-confidence labels cross the policy export boundary—analytics topics stay out of authorization.
-
Step 3
Ask the PDP
Send subject, action, resource, and context—identity chain, delegation status, requested model, intent labels, estimated cost, current spend state—to the AuthZEN-compatible PDP. The answer can be allow, deny, or allow with constraints.
-
Step 4
Enforce before inference
A denied request never reaches the provider. An allowed-with-constraints request is narrowed first: clamp maximum tokens, reject an unapproved model, or switch to a permitted model under policy.
-
Step 5
Attach the provider credential
Organization-managed credentials are resolved from approved vaults and applied only on the upstream provider connection. The client authenticates to the Identity Fabric and never holds the provider key.
-
Step 6
Relay the compatible request
OpenAI- and Anthropic-compatible routes carry the request body unchanged. Adoption means changing the endpoint and authentication—not rewriting application logic.
-
Step 7
Settle measured usage
After a completed allowed call, extract provider usage and settle the spend path. Analytics aggregates consumed spend and exposes it as policy information for the next decision.
-
Step 8
Create evidence
Completed allowed calls receive signed, hash-linked receipts binding policy context to measured usage—including the effective model actually used. Denied calls use decision telemetry, not receipt-backed execution claims.
The PDP is authoritative for business policy; the gateway is authoritative for enforcement at the model boundary. Every stage can narrow the permitted request—and no stage can widen what the PDP granted.
An identity-fabric enforcement point, not another proxy
A standalone gateway can add local rules. An identity-fabric gateway evaluates the same authoritative relationships and logical policy model used everywhere else in the enterprise.
| Typical AI gateway | EmpowerID LLM Gateway |
|---|---|
| API key or virtual key per client | Governed human, application, workload, or agent — with delegation checked at request time |
| Local rules configured in the proxy | The same logical PDP that governs human and workload access, via OpenID AuthZEN |
| Static filters and blocklists | High-confidence intent and data labels exported to policy; analytics topics kept separate |
| Token quotas and after-the-fact dashboards | Estimated cost and authoritative spend state evaluated in the decision, before the call |
| Config fallback or latency-based auto-router | Policy reroute, budget downgrade, provider failover — every candidate re-authorized |
| Gateway holds keys; clients hold gateway keys | Vault-backed organization keys; clients authenticate with Fabric sessions or PATs only |
| Operational logs and metrics | Signed, hash-linked receipts binding decision, constraints, effective model, and usage |
Budgets as policy, not metering
A rate limit answers how often a credential may be used. A budget policy answers whether a specific user, application, or delegated agent may consume a model under a time-bound financial boundary.
estimate authorize consume measure aggregate authorize again
- Stop consumption before a limit is exceeded
- Constrain maximum tokens for a subject or use case
- Permit a cheaper model when policy allows
- Attribute spend to the governed identity chain
- Keep financial controls consistent across applications
Authorization uses the pre-call estimate. A streamed response is not truncated mid-completion—a subject near a hard limit can overshoot by at most one completion’s difference between estimate and actual. Settlement records the overage; the next request is denied against it.
When the model changes, the policy does not
Most gateways substitute a model through config fallback or auto-router—substituting away authorization. In the EmpowerID design, every candidate model is re-evaluated by the PDP before it is used.
Policy reroute
Requested model denied for this subject or intent—the first PDP-authorized candidate from the cost-sorted allowed list serves the unchanged request.
Budget downgrade
Estimate breaches the spend boundary—cheaper allowed models are tried instead of a quota error. A use case pinned to one approved model is denied, never downgraded.
Provider failover
Provider errors or timeouts—circuit breakers move the request only where policy permits. Availability never overrides data, regional, or subject restrictions.
- A switch is never silent: the response declares when the effective model differs from the requested one, and the signed receipt records the model actually used.
- Switching applies to eligible chat routes—embeddings, audio, and image requests are not rerouted, and no switch occurs mid-stream once a response has begun.
One policy plane. Two specialized enforcement points.
A model invocation and an enterprise tool call have different risk shapes. The Identity Fabric keeps the decision shared and the enforcement specialized.
| Concern | LLM Gateway | MCP Gateway |
|---|---|---|
| Human, workload, and agent identity; delegation; shared PDP | Yes | Yes |
| Prompt classification; model and token constraints; AI budgets; model switching | Owner | — |
| Model-provider credential injection; model-call receipts | Owner | — |
| Tool discovery scoping; schema pins; parameter constraints | — | Owner |
| Enterprise connector credentials; tool-execution receipts | — | Owner |
Trustworthy failure, honest evidence
PDP outage
No silent bypass of authorization; any last-known-good window is bounded and explicit
Spend outage
A limited subject is never assumed to have unlimited funds — fail closed on hard boundaries
Classifier outage
Labels required by policy do not become an implicit allow when the classifier is down
Provider outage
Failover moves the request only to alternatives policy permits — never around policy
The PDP is on the hot path: a policy reroute across an allow-list of N candidates costs up to 1 + N evaluations for one user request. Entitlement caching for non-sensitive repeat decisions and deliberately narrow allow-lists keep that bounded.
The gateway governs the routed path; it cannot govern a developer calling a provider directly on a personal key. It belongs in a stack with egress allow-listing and provider-key governance that make the governed route the practical one—not merely the recommended one.
Signed, hash-linked receipts make alteration detectable within the retained chain—tamper-evident, not immutable. A receipt proves the recorded decision-and-usage path has cryptographic integrity; it does not certify that the model output was correct, safe, or compliant.
Five questions to ask any LLM gateway
- 1
Can it distinguish people, applications, workloads, and agents — and require active, scoped delegation behind an agent, rather than treating a key as the identity?
- 2
Is model authorization evaluated by a shared PDP with subject, intent, data, and spend context — or duplicated as local rules in each gateway?
- 3
Can a budget stop or constrain a request before provider consumption — and what happens when the authoritative spend state is unavailable?
- 4
Are prompt classifications separated into analytics and policy tiers — with the classifier advisory to policy, observable for drift, and never the business-decision engine?
- 5
Can a completed allowed call be linked to its policy decision, constraints, effective model, and measured usage in signed, tamper-evident form?
Built for the teams responsible for production AI
For security and iam
- Extend identity-aware policy to model access
- Require active delegation for agent subjects
- Reduce reliance on shared provider secrets
- Apply consistent controls across people, applications, and agents
- Trustworthy failure semantics when dependencies are unavailable
For ai platform and application teams
- Preserve familiar OpenAI- and Anthropic-compatible request patterns
- Centralize provider connectivity and credential management
- Separate policy logic from individual applications
- Multi-provider routing, circuit breaking, and policy-permitted failover
- Response signals when the effective model differs from the requested one
For finops
- Attribute model use to governed subjects
- Enforce policy before a provider call—not only report spend later
- Constrain tokens based on policy
- Permit lower-cost model alternatives where business rules allow
For risk and audit
- Link completed allowed use to a policy decision
- Retain signed, hash-linked evidence with honest deny semantics
- Investigate model activity through centralized analytics
- Support continuous control processes without claiming compliance by technology alone
Adopt governance without rewriting the application
The LLM Gateway exposes provider-compatible routes for common OpenAI and Anthropic request patterns. Applications authenticate through the Identity Fabric and point their SDK at the governed endpoint—the request body can remain unchanged while authorization, credentials, provider selection, budget policy, and evidence move into a shared control layer.
Key takeaways
- Routing, caching, and token limits are the baseline. The differentiating question is authority: who is acting, under whose delegation, for what intent, within what boundary.
- Enforce before inference — a denied request never reaches the provider, and an after-the-fact alert cannot recall data or cost.
- Spend is an authorization attribute — estimated before the call, measured after, and enforced before provider consumption rather than reported behind it.
- The client holds a Fabric identity, never the provider key. The PDP decides. The gateway enforces. The receipt remembers.
Common use cases
Govern enterprise copilots
Give internal copilots controlled access to approved providers and models without distributing provider keys to each application or browser client.
Control agent model consumption
Require an active identity chain and delegated authority before an agent invokes a model, then attribute usage to the governed subject.
Enforce intent-aware data policy
Use high-confidence prompt labels as policy inputs to distinguish a permitted general business question from a restricted request about an individual or protected data.
Stop spend before it happens
Evaluate current budget state before provider consumption, constrain requested tokens, or route to a lower-cost allowed model under policy.
Create reviewable evidence
Bind completed allowed model use to signed receipts and analytics so investigators can reconstruct the decision, constraints, effective model, and usage path.
Part of the EmpowerID Identity Fabric
Identity Provider and credentials
Authenticate principals, issue sessions, support identity chaining
Governed Authorization
PDP evaluation through AuthZEN-compatible interfaces
Identity graph and membership
Agent, user, delegation, and organization relationships
LLM Gateway
Model PEP—classification, budget, provider credentials, and allow-path receipts
MCP Gateway
Tool PEP at discovery and invocation; credential protection and routing
Agent Governance & Execution
Broader agent controls beyond model and tool boundaries
What LLM Gateway does not claim to solve
Credible AI governance requires defense in depth. LLM Gateway focuses on identity, authorization, spend, credentials, and allow-path evidence at the model boundary.
- Not a replacement for CDN, WAF, or general-purpose edge security
- Not a feature of the Identity Provider—it is a distinct Fabric execution service
- Not a tool-invocation gateway—that is the MCP Gateway’s enforcement domain
- Not output quarantine or universal response scanning in current public scope
- Not signed receipts for denied LLM calls—denials use decision and operational telemetry
- Not per-user bring-your-own-provider-key support in current public scope
- Not mid-stream truncation on default passthrough routes—pre-call estimate governs spend
- Not a guarantee of compliance by technology alone
Feature availability—including prompt classification depth, budget enforcement modes, model switching paths, provider routes, receipt coverage, circuit breaker behavior, and edition packaging—varies by release and deployment. Confirm scope with EmpowerID before customer-specific commitments.
FAQs
Is the LLM Gateway part of the EmpowerID IdP?
No. The IdP and LLM Gateway are distinct services within the EmpowerID Identity Fabric. The IdP establishes identity and delegation context. The policy decision point evaluates access. The LLM Gateway enforces the resulting decision at the model boundary.
How does prompt classification feed policy?
Inline classification derives intent and data context. Only high-precision intent labels on an allowlist enter PDP context; broad topic labels stay in analytics to prevent false-positive denials. Safety labels can fail closed at the gateway when configured; business rules are PDP decisions with structured advice.
How are AI budgets enforced?
Before the provider call, the gateway estimates cost and presents it with subject attribution to the PDP. Policy compares limits to authoritative consumed spend. Over-budget calls are denied before reaching the model. If spend state is unavailable, hard budgets fail closed for limited subjects.
Does it govern agent tool calls?
The LLM Gateway governs model calls. The EmpowerID MCP Gateway separately governs tool calls. The two enforcement points share identity and policy context on one policy plane.
Do applications need model-provider API keys?
No. Clients authenticate through the Identity Fabric. Organization-managed provider credentials are resolved at the gateway and used only on the provider connection.
Does every request receive a signed receipt?
Signed receipts are created for completed allowed calls. Denied LLM calls do not currently receive signed receipts and should be represented through decision and operational telemetry—not described as receipt-backed execution.
What happens when the requested model is denied?
Policy may reroute to another PDP-authorized candidate, downgrade to a cheaper allowed model when budget blocks the primary, or fail over to another permitted provider. Every substitute is re-authorized by policy—not substituted by configuration alone.
Does the gateway scan model output?
Today the gateway focuses on request classification, policy enforcement, model and token constraints, provider control, and allow-path receipts. Output quarantine and universal response scanning are not part of the current LLM Gateway scope.
Get Started
See a model call denied before inference
Request Demo See the platform in action
Talk to an Expert Technical consultation
