Platform - LLM Gateway

A drop-in AI gateway developers can use and security can govern.

Expose one OpenAI-compatible endpoint for approved providers and models. AI Warden routes requests, holds provider keys, enforces policies, calls DLP, compresses tokens, records governed decisions and gives platform teams the visibility they need to operate production AI.

View quickstartSecurity controls

Developer quickstart

Keep the OpenAI client. Change the base URL.

AI platform teams should not need every application team to rewrite their AI stack. AI Warden accepts OpenAI-compatible traffic, then applies enterprise routing, policy, cost and evidence controls at the gateway.

Before: direct provider calls

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY
});

Every app carries provider credentials, model names, logging decisions, retries and policy assumptions.

After: AI Warden Gateway

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AIW_TOKEN,
  baseURL: "https://aiw-gateway.company.com/v1"
});

Developers call approved model aliases while AI Warden handles key custody, routing, policy, DLP, budgets and logs.

curl https://aiw-gateway.company.com/v1/chat/completions   -H "Authorization: Bearer $AIW_TOKEN"   -H "Content-Type: application/json"   -d '{"model":"finance-gpt-4o-mini","messages":[{"role":"user","content":"Summarise this policy"}]}'

Provider and model routing

One approved model catalog instead of direct vendor sprawl.

AI Warden lets platform teams expose friendly model aliases while retaining control over the real provider, model, region, budget and policy behind each route.

  • Route to approved provider, cloud or self-hosted models through governed aliases.
  • Apply different policies by provider, model, user, team, agent, product or environment.
  • Use cheaper models for low-risk work and premium models only when policy or quality requires it.
  • Fallback, retry and rate-limit behavior can be governed at the gateway rather than scattered through clients.
model_aliases:
  finance-gpt-4o-mini:
    provider: azure-openai
    model: gpt-4o-mini
    policy: finance-standard
  risk-premium-review:
    provider: anthropic
    model: claude-sonnet
    policy: regulated-review
  local-redaction-test:
    provider: self-hosted
    model: llama-redaction
    policy: sandbox-only

Production observability

Debug the complete AI request path.

AI development leads need more than success/failure. They need to understand latency, token usage, provider errors, blocked requests, DLP events, model behavior and tool calls across production traffic.

TracePromptSee which route, scanner, ICAP service and policy version acted.
MeasureLatencyFind slow providers, failed routes, retries and blocked traffic.
AttributeUser + agentSeparate direct app traffic from delegated hosted-agent usage.
OptimizeCostCompare raw tokens, compressed tokens, avoided tokens and spend.

Guardrails and DLP

Security controls run inline, not after the incident.

AI Warden combines gateway policy, scanner rules and ICAP-supported DLP so unsafe content can be blocked before it reaches the model, tool or end user.

  • Prompt-injection, jailbreak, secrets, PII and custom-pattern scanning.
  • Request and response ICAP enforcement for enterprise DLP decisions.
  • Policy by user, team, agent, provider, model, route or product.
  • Actions including allow, flag, redact, modify, block, warn or session/tool containment.
StageControl
Before providerauth, model policy, budget, scanner, request inspection
During provider callroute, timeout, retry/fallback, token accounting
Before user responseresponse scanner, response inspection, redaction/block
After requestrequest log, analytics, budget update, governance event

AI FinOps

Compress tokens, govern demand and attribute every dollar.

LLM spend is not just a provider invoice. It is a runtime behavior problem. AI Warden sits in the path of governed calls so teams can reduce avoidable tokens, control demand and understand unit economics.

The cheapest token is the one you never send.

  • Context compaction for long-running chats and agents.
  • Retrieval payload slimming for RAG workloads.
  • Prompt normalization to remove repeated boilerplate.
  • Response-size controls to avoid expensive unused output.
  • Savings visibility for raw, compressed and avoided tokens.

Budgets at the level the business manages.

  • Budget by user, team, department, product, project, agent, route, provider or model.
  • Showback and chargeback exports for Finance.
  • Threshold actions: notify, require approval, downgrade, disable or block.
  • Cost per conversation, ticket, summary, tool run or agent workflow.

Built for platform operations

Give teams fast model access without losing production control.

AI Warden brings together OpenAI-compatible integration, provider routing, guardrails, DLP, request logs, budgets and governance workflows in a single enterprise gateway.

Request LLM Gateway demoSee AI FinOps