mirror of
https://github.com/ksyasuda/dotfiles.git
synced 2026-03-20 18:11:27 -07:00
update skills
This commit is contained in:
175
.agents/skills/cloudflare-deploy/references/ai-gateway/README.md
Normal file
175
.agents/skills/cloudflare-deploy/references/ai-gateway/README.md
Normal file
@@ -0,0 +1,175 @@
|
||||
# Cloudflare AI Gateway
|
||||
|
||||
Expert guidance for implementing Cloudflare AI Gateway - a universal gateway for AI model providers with analytics, caching, rate limiting, and routing capabilities.
|
||||
|
||||
## When to Use This Reference
|
||||
|
||||
- Setting up AI Gateway for any AI provider (OpenAI, Anthropic, Workers AI, etc.)
|
||||
- Implementing caching, rate limiting, or request retry/fallback
|
||||
- Configuring dynamic routing with A/B testing or model fallbacks
|
||||
- Managing provider API keys securely with BYOK
|
||||
- Adding security features (guardrails, DLP)
|
||||
- Setting up observability with logging and custom metadata
|
||||
- Debugging AI Gateway requests or optimizing configurations
|
||||
|
||||
## Quick Start
|
||||
|
||||
**What's your setup?**
|
||||
|
||||
- **Using Vercel AI SDK** → Pattern 1 (recommended) - see [sdk-integration.md](./sdk-integration.md)
|
||||
- **Using OpenAI SDK** → Pattern 2 - see [sdk-integration.md](./sdk-integration.md)
|
||||
- **Cloudflare Worker + Workers AI** → Pattern 3 - see [sdk-integration.md](./sdk-integration.md)
|
||||
- **Direct HTTP (any language)** → Pattern 4 - see [configuration.md](./configuration.md)
|
||||
- **Framework (LangChain, etc.)** → See [sdk-integration.md](./sdk-integration.md)
|
||||
|
||||
## Pattern 1: Vercel AI SDK (Recommended)
|
||||
|
||||
Most modern pattern using official `ai-gateway-provider` package with automatic fallbacks.
|
||||
|
||||
```typescript
|
||||
import { createAiGateway } from 'ai-gateway-provider';
|
||||
import { createOpenAI } from '@ai-sdk/openai';
|
||||
import { generateText } from 'ai';
|
||||
|
||||
const gateway = createAiGateway({
|
||||
accountId: process.env.CF_ACCOUNT_ID,
|
||||
gateway: process.env.CF_GATEWAY_ID,
|
||||
});
|
||||
|
||||
const openai = createOpenAI({
|
||||
apiKey: process.env.OPENAI_API_KEY
|
||||
});
|
||||
|
||||
// Single model
|
||||
const { text } = await generateText({
|
||||
model: gateway(openai('gpt-4o')),
|
||||
prompt: 'Hello'
|
||||
});
|
||||
|
||||
// Automatic fallback array
|
||||
const { text } = await generateText({
|
||||
model: gateway([
|
||||
openai('gpt-4o'), // Try first
|
||||
anthropic('claude-sonnet-4-5'), // Fallback
|
||||
]),
|
||||
prompt: 'Hello'
|
||||
});
|
||||
```
|
||||
|
||||
**Install:** `npm install ai-gateway-provider ai @ai-sdk/openai @ai-sdk/anthropic`
|
||||
|
||||
## Pattern 2: OpenAI SDK
|
||||
|
||||
Drop-in replacement for OpenAI API with multi-provider support.
|
||||
|
||||
```typescript
|
||||
import OpenAI from 'openai';
|
||||
|
||||
const client = new OpenAI({
|
||||
apiKey: process.env.OPENAI_API_KEY,
|
||||
baseURL: `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/compat`,
|
||||
defaultHeaders: {
|
||||
'cf-aig-authorization': `Bearer ${cfToken}` // For authenticated gateways
|
||||
}
|
||||
});
|
||||
|
||||
// Switch providers by changing model format: {provider}/{model}
|
||||
const response = await client.chat.completions.create({
|
||||
model: 'openai/gpt-4o', // or 'anthropic/claude-sonnet-4-5'
|
||||
messages: [{ role: 'user', content: 'Hello!' }]
|
||||
});
|
||||
```
|
||||
|
||||
## Pattern 3: Workers AI Binding
|
||||
|
||||
For Cloudflare Workers using Workers AI.
|
||||
|
||||
```typescript
|
||||
export default {
|
||||
async fetch(request, env, ctx) {
|
||||
const response = await env.AI.run(
|
||||
'@cf/meta/llama-3-8b-instruct',
|
||||
{ messages: [{ role: 'user', content: 'Hello!' }] },
|
||||
{
|
||||
gateway: {
|
||||
id: 'my-gateway',
|
||||
metadata: { userId: '123', team: 'engineering' }
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
return Response.json(response);
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
## Headers Quick Reference
|
||||
|
||||
| Header | Purpose | Example | Notes |
|
||||
|--------|---------|---------|-------|
|
||||
| `cf-aig-authorization` | Gateway auth | `Bearer {token}` | Required for authenticated gateways |
|
||||
| `cf-aig-metadata` | Tracking | `{"userId":"x"}` | Max 5 entries, flat structure |
|
||||
| `cf-aig-cache-ttl` | Cache duration | `3600` | Seconds, min 60, max 2592000 (30 days) |
|
||||
| `cf-aig-skip-cache` | Bypass cache | `true` | - |
|
||||
| `cf-aig-cache-key` | Custom cache key | `my-key` | Must be unique per response |
|
||||
| `cf-aig-collect-log` | Skip logging | `false` | Default: true |
|
||||
| `cf-aig-cache-status` | Cache hit/miss | Response only | `HIT` or `MISS` |
|
||||
|
||||
## In This Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [sdk-integration.md](./sdk-integration.md) | Vercel AI SDK, OpenAI SDK, Workers binding patterns |
|
||||
| [configuration.md](./configuration.md) | Dashboard setup, wrangler, API tokens |
|
||||
| [features.md](./features.md) | Caching, rate limits, guardrails, DLP, BYOK, unified billing |
|
||||
| [dynamic-routing.md](./dynamic-routing.md) | Fallbacks, A/B testing, conditional routing |
|
||||
| [troubleshooting.md](./troubleshooting.md) | Debugging, errors, observability, gotchas |
|
||||
|
||||
## Reading Order
|
||||
|
||||
| Task | Files |
|
||||
|------|-------|
|
||||
| First-time setup | README + [configuration.md](./configuration.md) |
|
||||
| SDK integration | README + [sdk-integration.md](./sdk-integration.md) |
|
||||
| Enable caching | README + [features.md](./features.md) |
|
||||
| Setup fallbacks | README + [dynamic-routing.md](./dynamic-routing.md) |
|
||||
| Debug errors | README + [troubleshooting.md](./troubleshooting.md) |
|
||||
|
||||
## Architecture
|
||||
|
||||
AI Gateway acts as a proxy between your application and AI providers:
|
||||
|
||||
```
|
||||
Your App → AI Gateway → AI Provider (OpenAI, Anthropic, etc.)
|
||||
↓
|
||||
Analytics, Caching, Rate Limiting, Logging
|
||||
```
|
||||
|
||||
**Key URL patterns:**
|
||||
- Unified API (OpenAI-compatible): `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions`
|
||||
- Provider-specific: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}/{endpoint}`
|
||||
- Dynamic routes: Use route name instead of model: `dynamic/{route-name}`
|
||||
|
||||
## Gateway Types
|
||||
|
||||
1. **Unauthenticated Gateway**: Open access (not recommended for production)
|
||||
2. **Authenticated Gateway**: Requires `cf-aig-authorization` header with Cloudflare API token (recommended)
|
||||
|
||||
## Provider Authentication Options
|
||||
|
||||
1. **Unified Billing**: Use AI Gateway billing to pay for inference (keyless mode - no provider API key needed)
|
||||
2. **BYOK (Store Keys)**: Store provider API keys in Cloudflare dashboard
|
||||
3. **Request Headers**: Include provider API key in each request
|
||||
|
||||
## Related Skills
|
||||
|
||||
- [Workers AI](../workers-ai/README.md) - For `env.AI.run()` details
|
||||
- [Agents SDK](../agents-sdk/README.md) - For stateful AI patterns
|
||||
- [Vectorize](../vectorize/README.md) - For RAG patterns with embeddings
|
||||
|
||||
## Resources
|
||||
|
||||
- [Official Docs](https://developers.cloudflare.com/ai-gateway/)
|
||||
- [API Reference](https://developers.cloudflare.com/api/resources/ai_gateway/)
|
||||
- [Provider Guides](https://developers.cloudflare.com/ai-gateway/usage/providers/)
|
||||
- [Discord Community](https://discord.cloudflare.com)
|
||||
Reference in New Issue
Block a user