mirror of
https://github.com/ksyasuda/dotfiles.git
synced 2026-03-21 06:11:27 -07:00
update skills
This commit is contained in:
@@ -0,0 +1,78 @@
|
||||
# Cloudflare Browser Rendering Skill Reference
|
||||
|
||||
**Description**: Expert knowledge for Cloudflare Browser Rendering - control headless Chrome on Cloudflare's global network for browser automation, screenshots, PDFs, web scraping, testing, and content generation.
|
||||
|
||||
**When to use**: Any task involving Cloudflare Browser Rendering including: taking screenshots, generating PDFs, web scraping, browser automation, testing web applications, extracting structured data, capturing page metrics, or automating browser interactions.
|
||||
|
||||
## Decision Tree
|
||||
|
||||
### REST API vs Workers Bindings
|
||||
|
||||
**Use REST API when:**
|
||||
- One-off, stateless tasks (screenshot, PDF, content fetch)
|
||||
- No Workers infrastructure yet
|
||||
- Simple integrations from external services
|
||||
- Need quick prototyping without deployment
|
||||
|
||||
**Use Workers Bindings when:**
|
||||
- Complex browser automation workflows
|
||||
- Need session reuse for performance
|
||||
- Multiple page interactions per request
|
||||
- Custom scripting and logic required
|
||||
- Building production applications
|
||||
|
||||
### Puppeteer vs Playwright
|
||||
|
||||
| Feature | Puppeteer | Playwright |
|
||||
|---------|-----------|------------|
|
||||
| API Style | Chrome DevTools Protocol | High-level abstractions |
|
||||
| Selectors | CSS, XPath | CSS, text, role, test-id |
|
||||
| Best for | Advanced control, CDP access | Quick automation, testing |
|
||||
| Learning curve | Steeper | Gentler |
|
||||
|
||||
**Use Puppeteer:** Need CDP protocol access, Chrome-specific features, migration from existing Puppeteer code
|
||||
**Use Playwright:** Modern selector APIs, cross-browser patterns, faster development
|
||||
|
||||
## Tier Limits Summary
|
||||
|
||||
| Limit | Free Tier | Paid Tier |
|
||||
|-------|-----------|-----------|
|
||||
| Daily browser time | 10 minutes | Unlimited* |
|
||||
| Concurrent sessions | 3 | 30 |
|
||||
| Requests per minute | 6 | 180 |
|
||||
|
||||
*Subject to fair-use policy. See [gotchas.md](gotchas.md) for details.
|
||||
|
||||
## Reading Order
|
||||
|
||||
**New to Browser Rendering:**
|
||||
1. [configuration.md](configuration.md) - Setup and deployment
|
||||
2. [patterns.md](patterns.md) - Common use cases with examples
|
||||
3. [api.md](api.md) - API reference
|
||||
4. [gotchas.md](gotchas.md) - Avoid common pitfalls
|
||||
|
||||
**Specific task:**
|
||||
- **Setup/deployment** → [configuration.md](configuration.md)
|
||||
- **API reference/endpoints** → [api.md](api.md)
|
||||
- **Example code/patterns** → [patterns.md](patterns.md)
|
||||
- **Debugging/troubleshooting** → [gotchas.md](gotchas.md)
|
||||
|
||||
**REST API users:**
|
||||
- Start with [api.md](api.md) REST API section
|
||||
- Check [gotchas.md](gotchas.md) for rate limits
|
||||
|
||||
**Workers users:**
|
||||
- Start with [configuration.md](configuration.md)
|
||||
- Review [patterns.md](patterns.md) for session management
|
||||
- Reference [api.md](api.md) for Workers Bindings
|
||||
|
||||
## In This Reference
|
||||
|
||||
- **[configuration.md](configuration.md)** - Setup, deployment, wrangler config, compatibility
|
||||
- **[api.md](api.md)** - REST API endpoints + Workers Bindings (Puppeteer/Playwright)
|
||||
- **[patterns.md](patterns.md)** - Common patterns, use cases, real examples
|
||||
- **[gotchas.md](gotchas.md)** - Troubleshooting, best practices, tier limits, common errors
|
||||
|
||||
## See Also
|
||||
|
||||
- [Cloudflare Docs](https://developers.cloudflare.com/browser-rendering/)
|
||||
@@ -0,0 +1,108 @@
|
||||
# Browser Rendering API
|
||||
|
||||
## REST API
|
||||
|
||||
**Base:** `https://api.cloudflare.com/client/v4/accounts/{accountId}/browser-rendering`
|
||||
**Auth:** `Authorization: Bearer <token>` (Browser Rendering - Edit permission)
|
||||
|
||||
### Endpoints
|
||||
|
||||
| Endpoint | Description | Key Options |
|
||||
|----------|-------------|-------------|
|
||||
| `/content` | Get rendered HTML | `url`, `waitUntil` |
|
||||
| `/screenshot` | Capture image | `screenshotOptions: {type, fullPage, clip}` |
|
||||
| `/pdf` | Generate PDF | `pdfOptions: {format, landscape, margin}` |
|
||||
| `/snapshot` | HTML + inlined resources | `url` |
|
||||
| `/scrape` | Extract by selectors | `selectors: ["h1", ".price"]` |
|
||||
| `/json` | AI-structured extraction | `schema: {name: "string", price: "number"}` |
|
||||
| `/links` | Get all links | `url` |
|
||||
| `/markdown` | Convert to markdown | `url` |
|
||||
|
||||
```bash
|
||||
curl -X POST '.../browser-rendering/screenshot' \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-d '{"url":"https://example.com","screenshotOptions":{"fullPage":true}}'
|
||||
```
|
||||
|
||||
## Workers Binding
|
||||
|
||||
```jsonc
|
||||
// wrangler.jsonc
|
||||
{ "browser": { "binding": "MYBROWSER" } }
|
||||
```
|
||||
|
||||
## Puppeteer
|
||||
|
||||
```typescript
|
||||
import puppeteer from "@cloudflare/puppeteer";
|
||||
|
||||
const browser = await puppeteer.launch(env.MYBROWSER, { keep_alive: 600000 });
|
||||
const page = await browser.newPage();
|
||||
await page.goto('https://example.com', { waitUntil: 'networkidle0' });
|
||||
|
||||
// Content
|
||||
const html = await page.content();
|
||||
const title = await page.title();
|
||||
|
||||
// Screenshot/PDF
|
||||
await page.screenshot({ fullPage: true, type: 'png' });
|
||||
await page.pdf({ format: 'A4', printBackground: true });
|
||||
|
||||
// Interaction
|
||||
await page.click('#button');
|
||||
await page.type('#input', 'text');
|
||||
await page.evaluate(() => document.querySelector('h1')?.textContent);
|
||||
|
||||
// Session management
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
## Playwright
|
||||
|
||||
```typescript
|
||||
import { launch, connect } from "@cloudflare/playwright";
|
||||
|
||||
const browser = await launch(env.MYBROWSER, { keep_alive: 600000 });
|
||||
const page = await browser.newPage();
|
||||
|
||||
await page.goto('https://example.com', { waitUntil: 'networkidle' });
|
||||
|
||||
// Modern selectors
|
||||
await page.locator('.button').click();
|
||||
await page.getByText('Submit').click();
|
||||
await page.getByTestId('search').fill('query');
|
||||
|
||||
// Context for isolation
|
||||
const context = await browser.newContext({
|
||||
viewport: { width: 1920, height: 1080 },
|
||||
userAgent: 'custom'
|
||||
});
|
||||
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
## Session Management
|
||||
|
||||
```typescript
|
||||
// List sessions
|
||||
await puppeteer.sessions(env.MYBROWSER);
|
||||
|
||||
// Connect to existing
|
||||
await puppeteer.connect(env.MYBROWSER, sessionId);
|
||||
|
||||
// Check limits
|
||||
await puppeteer.limits(env.MYBROWSER);
|
||||
// { remaining: ms, total: ms, concurrent: n }
|
||||
```
|
||||
|
||||
## Key Options
|
||||
|
||||
| Option | Values |
|
||||
|--------|--------|
|
||||
| `waitUntil` | `load`, `domcontentloaded`, `networkidle0`, `networkidle2` |
|
||||
| `keep_alive` | Max 600000ms (10 min) |
|
||||
| `screenshot.type` | `png`, `jpeg` |
|
||||
| `pdf.format` | `A4`, `Letter`, `Legal` |
|
||||
@@ -0,0 +1,78 @@
|
||||
# Configuration & Setup
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
npm install @cloudflare/puppeteer # or @cloudflare/playwright
|
||||
```
|
||||
|
||||
**Use Cloudflare packages** - standard `puppeteer`/`playwright` won't work in Workers.
|
||||
|
||||
## wrangler.json
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "browser-worker",
|
||||
"main": "src/index.ts",
|
||||
"compatibility_date": "2025-01-01",
|
||||
"compatibility_flags": ["nodejs_compat"],
|
||||
"browser": {
|
||||
"binding": "MYBROWSER"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Required:** `nodejs_compat` flag and `browser.binding`.
|
||||
|
||||
## TypeScript
|
||||
|
||||
```typescript
|
||||
interface Env {
|
||||
MYBROWSER: Fetcher;
|
||||
}
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Env): Promise<Response> {
|
||||
// ...
|
||||
}
|
||||
} satisfies ExportedHandler<Env>;
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
wrangler dev --remote # --remote required for browser binding
|
||||
```
|
||||
|
||||
**Local mode does NOT support Browser Rendering** - must use `--remote`.
|
||||
|
||||
## REST API
|
||||
|
||||
No wrangler config needed. Get API token with "Browser Rendering - Edit" permission.
|
||||
|
||||
```bash
|
||||
curl -X POST \
|
||||
'https://api.cloudflare.com/client/v4/accounts/{accountId}/browser-rendering/screenshot' \
|
||||
-H 'Authorization: Bearer TOKEN' \
|
||||
-d '{"url": "https://example.com"}' --output screenshot.png
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
| Requirement | Value |
|
||||
|-------------|-------|
|
||||
| Node.js compatibility | `nodejs_compat` flag |
|
||||
| Compatibility date | 2023-03-01+ |
|
||||
| Module format | ES modules only |
|
||||
| Browser | Chromium 119+ (no Firefox/Safari) |
|
||||
|
||||
**Not supported:** WebGL, WebRTC, extensions, `file://` protocol, Service Worker syntax.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Error | Solution |
|
||||
|-------|----------|
|
||||
| `MYBROWSER is undefined` | Use `wrangler dev --remote` |
|
||||
| `nodejs_compat not enabled` | Add to `compatibility_flags` |
|
||||
| `Module not found` | `npm install @cloudflare/puppeteer` |
|
||||
| `Browser Rendering not available` | Enable in dashboard |
|
||||
@@ -0,0 +1,88 @@
|
||||
# Browser Rendering Gotchas
|
||||
|
||||
## Tier Limits
|
||||
|
||||
| Limit | Free | Paid |
|
||||
|-------|------|------|
|
||||
| Daily browser time | 10 min | Unlimited* |
|
||||
| Concurrent sessions | 3 | 30 |
|
||||
| Requests/minute | 6 | 180 |
|
||||
| Session keep-alive | 10 min max | 10 min max |
|
||||
|
||||
*Subject to fair-use policy.
|
||||
|
||||
**Check quota:**
|
||||
```typescript
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
// { remaining: 540000, total: 600000, concurrent: 2 }
|
||||
```
|
||||
|
||||
## Always Close Browsers
|
||||
|
||||
```typescript
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
try {
|
||||
const page = await browser.newPage();
|
||||
await page.goto("https://example.com");
|
||||
return new Response(await page.content());
|
||||
} finally {
|
||||
await browser.close(); // ALWAYS in finally
|
||||
}
|
||||
```
|
||||
|
||||
**Workers vs REST:** REST auto-closes after timeout. Workers must call `close()` or session stays open until `keep_alive` expires.
|
||||
|
||||
## Optimize Concurrency
|
||||
|
||||
```typescript
|
||||
// ❌ 3 sessions (hits free tier limit)
|
||||
const browser1 = await puppeteer.launch(env.MYBROWSER);
|
||||
const browser2 = await puppeteer.launch(env.MYBROWSER);
|
||||
|
||||
// ✅ 1 session, multiple pages
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
const page1 = await browser.newPage();
|
||||
const page2 = await browser.newPage();
|
||||
```
|
||||
|
||||
## Common Errors
|
||||
|
||||
| Error | Cause | Fix |
|
||||
|-------|-------|-----|
|
||||
| Session limit exceeded | Too many concurrent | Close unused browsers, use pages not browsers |
|
||||
| Page navigation timeout | Slow page or `networkidle` on busy page | Increase timeout, use `waitUntil: "load"` |
|
||||
| Session not found | Expired session | Catch error, launch new session |
|
||||
| Evaluation failed | DOM element missing | Use `?.` optional chaining |
|
||||
| Protocol error: Target closed | Page closed during operation | Await all ops before closing |
|
||||
|
||||
## page.evaluate() Gotchas
|
||||
|
||||
```typescript
|
||||
// ❌ Outer scope not available
|
||||
const selector = "h1";
|
||||
await page.evaluate(() => document.querySelector(selector));
|
||||
|
||||
// ✅ Pass as argument
|
||||
await page.evaluate((sel) => document.querySelector(sel)?.textContent, selector);
|
||||
```
|
||||
|
||||
## Performance
|
||||
|
||||
**waitUntil options (fastest to slowest):**
|
||||
1. `domcontentloaded` - DOM ready
|
||||
2. `load` - load event (default)
|
||||
3. `networkidle0` - no network for 500ms
|
||||
|
||||
**Block unnecessary resources:**
|
||||
```typescript
|
||||
await page.setRequestInterception(true);
|
||||
page.on("request", (req) => {
|
||||
if (["image", "stylesheet", "font"].includes(req.resourceType())) {
|
||||
req.abort();
|
||||
} else {
|
||||
req.continue();
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
**Session reuse:** Cold start ~1-2s, warm connect ~100-200ms. Store sessionId in KV for reuse.
|
||||
@@ -0,0 +1,91 @@
|
||||
# Browser Rendering Patterns
|
||||
|
||||
## Basic Worker
|
||||
|
||||
```typescript
|
||||
import puppeteer from "@cloudflare/puppeteer";
|
||||
|
||||
export default {
|
||||
async fetch(request, env) {
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
try {
|
||||
const page = await browser.newPage();
|
||||
await page.goto("https://example.com");
|
||||
return new Response(await page.content());
|
||||
} finally {
|
||||
await browser.close(); // ALWAYS in finally
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
## Session Reuse
|
||||
|
||||
Keep sessions alive for performance:
|
||||
```typescript
|
||||
let sessionId = await env.SESSION_KV.get("browser-session");
|
||||
if (sessionId) {
|
||||
browser = await puppeteer.connect(env.MYBROWSER, sessionId);
|
||||
} else {
|
||||
browser = await puppeteer.launch(env.MYBROWSER, { keep_alive: 600000 });
|
||||
await env.SESSION_KV.put("browser-session", browser.sessionId(), { expirationTtl: 600 });
|
||||
}
|
||||
// Don't close browser to keep session alive
|
||||
```
|
||||
|
||||
## Common Operations
|
||||
|
||||
| Task | Code |
|
||||
|------|------|
|
||||
| Screenshot | `await page.screenshot({ type: "png", fullPage: true })` |
|
||||
| PDF | `await page.pdf({ format: "A4", printBackground: true })` |
|
||||
| Extract data | `await page.evaluate(() => document.querySelector('h1').textContent)` |
|
||||
| Fill form | `await page.type('#input', 'value'); await page.click('button')` |
|
||||
| Wait nav | `await Promise.all([page.waitForNavigation(), page.click('a')])` |
|
||||
|
||||
## Parallel Scraping
|
||||
|
||||
```typescript
|
||||
const pages = await Promise.all(urls.map(() => browser.newPage()));
|
||||
await Promise.all(pages.map((p, i) => p.goto(urls[i])));
|
||||
const titles = await Promise.all(pages.map(p => p.title()));
|
||||
```
|
||||
|
||||
## Playwright Selectors
|
||||
|
||||
```typescript
|
||||
import { launch } from "@cloudflare/playwright";
|
||||
const browser = await launch(env.MYBROWSER);
|
||||
await page.getByRole("button", { name: "Sign in" }).click();
|
||||
await page.getByLabel("Email").fill("user@example.com");
|
||||
await page.getByTestId("submit-button").click();
|
||||
```
|
||||
|
||||
## Incognito Contexts
|
||||
|
||||
Isolated sessions without multiple browsers:
|
||||
```typescript
|
||||
const ctx1 = await browser.createIncognitoBrowserContext();
|
||||
const ctx2 = await browser.createIncognitoBrowserContext();
|
||||
// Each has isolated cookies/storage
|
||||
```
|
||||
|
||||
## Quota Check
|
||||
|
||||
```typescript
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
if (limits.remaining < 60000) return new Response("Quota low", { status: 429 });
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```typescript
|
||||
try {
|
||||
await page.goto(url, { timeout: 30000, waitUntil: "networkidle0" });
|
||||
} catch (e) {
|
||||
if (e.message.includes("timeout")) return new Response("Timeout", { status: 504 });
|
||||
if (e.message.includes("Session limit")) return new Response("Too many sessions", { status: 429 });
|
||||
} finally {
|
||||
if (browser) await browser.close();
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user