---
title: "Cloudflare AI Gateway - AI Application Control Plane"
description: "An intelligent control plane for your AI applications. Connect to any model, dynamic routing, caching, observability, and unified billing for AI workloads."
url: "https://www.cloudflare.com/products/ai-gateway"
---

# AI Gateway

> Connect to any model, dynamically route requests, and manage usage, billing, and logs from one unified gateway.

## Key Features

- Multi-provider support
- Response caching
- Fallback routing
- Rate limiting
- Usage analytics
- Token counting
- Unified billing

## Benefits

### Reduce Costs & Latency

Easily cache responses and reduces redundant API calls — leading to direct cost savings.

### Improve Reliability with Dynamic Controls

Configure how and when model providers APIs are called based on specific attributes or fallbacks

### Add Observability

Enables rich usage insights such as token counts, prompt performance, and pattern analysis.

## Use Cases

### Reducing latency and cost of AI apps by caching API responses

Optimize your AI application performance and reduce costs by intelligently caching responses from AI providers.

### Usage analytics — monitoring prompt performance, token counts, and behavior

Gain deep insights into your AI usage patterns, token consumption, and prompt performance across all providers.

### Building custom dashboards and alerting systems directly from AI Gateway logs

Create comprehensive monitoring and alerting systems using AI Gateway's rich logging and metrics data.

## Code Examples

### One-line setup

Set up AI Gateway to proxy requests to AI providers with caching and observability. Compatible with OpenAI SDK and AI SDK.

```javascript
// wrangler.jsonc
// Simple configuration
{
  "ai": {
    "binding": "AI"
  }
}

// Pass through the Gateway from your Worker with Workers AI
// index.js
const resp = await env.AI.run(
  "@cf/meta/llama-3.1-8b-instruct",
  {
    prompt: "tell me a joke",
  },
  {
    gateway: {
      id: "my-gateway",
    },
  },
);

// Use with OpenAI SDK
import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: "my api key", // defaults to process.env["OPENAI_API_KEY"]
  baseURL: await env.AI.gateway("my-gateway").getUrl("openai"),
});


```

### Caching AI Responses

Configure intelligent caching to reduce costs and improve response times.

```javascript
// AI Gateway with caching set
env.AI.gateway('my-gateway').run({
   provider: "openai",
   endpoint: "gpt-3.5-turbo",
   headers: {
       authorization: "Bearer my-api-token",
       "cf-aig-cache-ttl": 3600
   },
   query: {
       messages: [
           {role: 'user', content: 'What is the capital of France?'}
       ],
   },
})

```

### Send feedback & access logs

Your AI Gateway dashboard shows logs of individual requests, including the user prompt, model response, provider, timestamp, request status, token usage, cost, and duration. 

```logging

// The patchLog method allows you to send feedback, score, and metadata for a specific log ID. All object properties are optional, so you can include any combination of the parameters:
gateway.patchLog("my-log-id", {
  feedback: 1,
  score: 100,
  metadata: {
    user: "123",
  },
});

// Read log details in your Worker with getLog
const log = await gateway.getLog("my-log-id");
        
```

### Fallback Routing and Rate Limiting

Configure intelligent fallbacks and rate limiting for reliable AI operations.

```fallback
// Add as many fallbacks as you need, just by adding another object in the array.

curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id} \
  --header 'Content-Type: application/json' \
  --data '[
  {
    "provider": "workers-ai",
    "endpoint": "@cf/meta/llama-3.1-8b-instruct",
    "headers": {
      "Authorization": "Bearer {cloudflare_token}",
      "Content-Type": "application/json"
    },
    "query": {
      "messages": [
        {
          "role": "system",
          "content": "You are a friendly assistant"
        },
        {
          "role": "user",
          "content": "What is Cloudflare?"
        }
      ]
    }
  },
  {
    "provider": "openai",
    "endpoint": "chat/completions",
    "headers": {
      "Authorization": "Bearer {open_ai_token}",
      "Content-Type": "application/json"
    },
    "query": {
      "model": "gpt-4o-mini",
      "stream": true,
      "messages": [
        {
          "role": "user",
          "content": "What is Cloudflare?"
        }
      ]
    }
  }
]'
```

## Resources

- [Full Documentation](https://developers.cloudflare.com/ai-gateway): Complete technical documentation
- [Get Started](https://dash.cloudflare.com/sign-up): Sign up and start building
- [Pricing](/plans.md): See pricing details

## Related Products

- [Agents](/products/agents.md): Build stateful AI agents
- [AI Search](/products/ai-search.md): Instant retrieval
- [Vectorize](/products/vectorize.md): Vector database
- [Web3](/products/web3.md): Web3 Infrastructure

---

*This is a markdown version of [https://www.cloudflare.com/products/ai-gateway](https://www.cloudflare.com/products/ai-gateway) for AI/LLM consumption.*