AI gateway: DLP and outbound data control

*Article from our engineering series on enterprise AI, legacy integration and information governance.*

Where does what your employees type into AI actually end up

Every executive has had this uncomfortable thought at some point: an analyst pastes a contract excerpt into a public chatbot to "quickly summarize it," a salesperson asks for help answering a client by pasting an entire email thread, a developer sends a code snippet with an embedded API key. None of this goes through approval, none of it is logged, and none of it is reversible once the text leaves the corporate network.

The engineering answer to this problem is not an acceptable use policy printed and signed — it is a single point of egress. An AI gateway is the component that intercepts every call to language models and applies identity, redaction of sensitive data, cost limits and logging before anything leaves the corporate network. Without this component, "AI governance" is a sentence in a document; with it, it is a verifiable property of the system.

Why blocking domains at the proxy is not enough

The initial temptation is to solve this at the firewall: block chatgpt.com, gemini.google.com and similar destinations at the internet egress. This fails for three reasons. First, the list of AI services grows faster than any blocklist can keep up with. Second, legitimate teams need access to models for internal automations — blocking everything paralyzes the business. Third, and most important: network blocking decides nothing about the content of the request. It prevents the wrong destination, but doesn't address the real problem, which is sensitive data leaving anyway.

The gateway solves this by inverting the logic: instead of banning destinations, the company offers a single approved path with built-in controls, and makes that path easier to use than any alternative. Shadow AI is defeated with convenience, not just rules.

Anatomy of the gateway

Application/User
      |
      v
[ AI Gateway ]
   |  |  |  |
   |  |  |  +--> Logging and retention (audit)
   |  |  +-----> Rate limit / cost quota per team
   |  +--------> DLP (request and response)
   +-----------> Routing by use case / model
      |
      v
Model providers (internal, OpenAI, Anthropic, Azure, etc.)

The gateway sits in the path of every call, whether made by an internal chatbot, an RPA automation, or an IDE extension. No system calls a model provider directly — everyone calls the gateway, and the gateway decides the rest.

Authentication and identity propagation

Each request reaches the gateway with an identity token from the calling user or service — typically a JWT issued by the corporate identity provider (Azure AD, Okta). The gateway validates that token, extracts relevant claims (team, role, data access level) and propagates that identity forward, both for policy decisions and for the audit log. This is what lets you answer, with evidence, "who requested this and with what access profile."

{
  "sub": "user:ana.silva",
  "team": "finance",
  "data_clearance": "confidential",
  "cost_center": "CC-4021"
}

Without identity propagation, every gateway log becomes a list of anonymous calls — auditable in volume, useless for individual accountability.

Routing by use case and by model

Not every request should go to the same model. A summary of an internal document can go to a cheaper model; a sensitive legal analysis may require a model hosted in an environment that doesn't retain data for training; a use case classified as high risk may require human approval before it goes out.

Routing is configuration, not code scattered across applications. This lets you switch providers, adjust risk policy, or react to a security incident from a single place.

DLP and redaction on request and response

This is the core of outbound control. The gateway applies two inspection points:

1. 1.On the request (before it goes out to the provider): detects and redacts sensitive patterns — national ID numbers, card numbers, API keys, customer emails, terms from a business-specific sensitive data dictionary (M&A project names, contract numbers). Redacted data can be replaced with reversible tokens that only resolve inside the company's perimeter.
2. 2.On the response (before returning it to the user): checks whether the model "leaked" something it shouldn't — for example, echoing back a sensitive value that was in the context, or generating content that matches prohibited patterns (malicious code, another customer's data due to context contamination).

def process_request(payload, policy):
    redacted, matches = dlp.redact(payload.text, policy.patterns)
    if matches and policy.action == "block":
        raise BlockedRequest(matches)
    response = model_provider.call(redacted)
    return dlp.redact(response.text, policy.patterns)

Bidirectional redaction is what differentiates a security gateway from a simple cost proxy. Without it, the original problem — sensitive data leaving the company — still exists, it just now passes through a nicer, logged place.

Rate and cost limits per team

Each team or application receives a quota of requests and token spend. This prevents two distinct problems: a looping bug generating thousands of calls in minutes, and a team burning the whole quarter's AI budget in a week without anyone noticing. The gateway exposes this consumption in real time, by cost center, which also solves the FinOps pain of "who is spending what" without waiting on the provider's monthly invoice.

Blocking shadow AI

At the network edge, the gateway can be combined with a policy that denies, by default, any outbound traffic to AI domains other than the gateway. Corporate proxy tools (CASB) help identify attempts to use unapproved services, but the practical effect only holds if the approved path is fast and not bureaucratic — if requesting gateway access takes weeks, employees go back to using a personal browser.

Logging and retention

Every event — request, response, DLP decision, calling identity, model used, cost — is logged in a structured way and retained according to the company's data policy (typically 1 to 7 years, depending on the regulated sector). This log is what turns "we trust nobody leaked data" into "we have evidence of what went out, when, and where."

What to do on Monday

1. 1.Map how many access points to AI models exist today in the company — chatbots, IDE extensions, automations, direct API integrations — and count how many go through a central control.
2. 2.Choose an AI gateway (commercial or open source, such as LiteLLM, Portkey, or a custom layer) and put at least one high-risk use case behind it this week.
3. 3.Define, with information security, the three to five most critical data patterns for redaction (national IDs, card numbers, API keys, customer names) and implement the DLP rule on the request before handling the response.
4. 4.Set up per-team cost alerts with a daily limit, not just monthly — the damage from a runaway loop is measured in hours.
5. 5.Publish the approved path visibly and make it fast to use, so the easiest alternative is the controlled one, not the shadow one.

Further reading

Executive track:

• Where what your employees type into AI ends up — the business view of this same topic.