mirror of
https://github.com/ksyasuda/dotfiles.git
synced 2026-03-22 06:11:27 -07:00
update skills
This commit is contained in:
@@ -0,0 +1,65 @@
|
||||
# Cloudflare Realtime SFU Reference
|
||||
|
||||
Expert guidance for building real-time audio/video/data applications using Cloudflare Realtime SFU (Selective Forwarding Unit).
|
||||
|
||||
## Reading Order
|
||||
|
||||
| Task | Files | ~Tokens |
|
||||
|------|-------|---------|
|
||||
| New project | README → configuration | ~1200 |
|
||||
| Implement publish/subscribe | README → api | ~1600 |
|
||||
| Add PartyTracks | patterns (PartyTracks section) | ~800 |
|
||||
| Build presence system | patterns (DO section) | ~800 |
|
||||
| Debug connection issues | gotchas | ~700 |
|
||||
| Scale to millions | patterns (Cascading section) | ~600 |
|
||||
| Add simulcast | patterns (Advanced section) | ~500 |
|
||||
| Configure TURN | configuration (TURN section) | ~400 |
|
||||
|
||||
## In This Reference
|
||||
|
||||
- **[configuration.md](configuration.md)** - Setup, deployment, environment variables, Wrangler config
|
||||
- **[api.md](api.md)** - Sessions, tracks, endpoints, request/response patterns
|
||||
- **[patterns.md](patterns.md)** - Architecture patterns, use cases, integration examples
|
||||
- **[gotchas.md](gotchas.md)** - Common issues, debugging, performance, security
|
||||
|
||||
## Quick Start
|
||||
|
||||
Cloudflare Realtime SFU: WebRTC infrastructure on global network (310+ cities). Anycast routing, no regional constraints, pub/sub model.
|
||||
|
||||
**Core concepts:**
|
||||
- **Sessions:** WebRTC PeerConnection to Cloudflare edge
|
||||
- **Tracks:** Audio/video/data channels you publish or subscribe to
|
||||
- **No rooms:** Build presence layer yourself via track sharing (see patterns.md)
|
||||
|
||||
**Mental model:** Your client establishes one WebRTC session, publishes tracks (audio/video), shares track IDs via your backend, others subscribe to your tracks using track IDs + your session ID.
|
||||
|
||||
## Choose Your Approach
|
||||
|
||||
| Approach | When to Use | Complexity |
|
||||
|----------|-------------|------------|
|
||||
| **PartyTracks** | Production apps with device switching, React | Low - Observable-based, handles reconnections |
|
||||
| **Raw API** | Custom requirements, non-browser, learning | Medium - Full control, manual WebRTC lifecycle |
|
||||
| **RealtimeKit** | End-to-end SDK with UI components | Lowest - Managed state, React hooks |
|
||||
|
||||
**Recommendation:** Start with PartyTracks for most production applications. See patterns.md for PartyTracks examples.
|
||||
|
||||
## SFU vs RealtimeKit
|
||||
|
||||
- **Realtime SFU:** WebRTC infrastructure (this reference). Build your own signaling, presence, UI.
|
||||
- **RealtimeKit:** SDK layer on top of SFU. Includes React hooks, state management, UI components. Part of Cloudflare AI platform.
|
||||
|
||||
Use SFU directly when you need custom signaling or non-React framework. Use RealtimeKit for faster development with React.
|
||||
|
||||
## Setup
|
||||
|
||||
Dashboard: https://dash.cloudflare.com/?to=/:account/calls
|
||||
|
||||
Get `CALLS_APP_ID` and `CALLS_APP_SECRET` from dashboard, then see configuration.md for deployment.
|
||||
|
||||
## See Also
|
||||
|
||||
- [Orange Meets Demo](https://demo.orange.cloudflare.dev/)
|
||||
- [Orange Source](https://github.com/cloudflare/orange)
|
||||
- [Calls Examples](https://github.com/cloudflare/calls-examples)
|
||||
- [API Reference](https://developers.cloudflare.com/api/resources/calls/)
|
||||
- [RealtimeKit Docs](https://developers.cloudflare.com/workers-ai/realtimekit/)
|
||||
158
.agents/skills/cloudflare-deploy/references/realtime-sfu/api.md
Normal file
158
.agents/skills/cloudflare-deploy/references/realtime-sfu/api.md
Normal file
@@ -0,0 +1,158 @@
|
||||
# API Reference
|
||||
|
||||
## Authentication
|
||||
|
||||
```bash
|
||||
curl -X POST 'https://rtc.live/v1/apps/${CALLS_APP_ID}/sessions/new' \
|
||||
-H "Authorization: Bearer ${CALLS_APP_SECRET}"
|
||||
```
|
||||
|
||||
## Core Concepts
|
||||
|
||||
**Sessions:** PeerConnection to Cloudflare edge
|
||||
**Tracks:** Media/data channels (audio/video/datachannel)
|
||||
**No rooms:** Build presence via track sharing
|
||||
|
||||
## Client Libraries
|
||||
|
||||
**PartyTracks (Recommended):** Observable-based client library for production use. Handles device changes, network switches, ICE restarts automatically. Push/pull API with React hooks. See patterns.md for full examples.
|
||||
|
||||
```bash
|
||||
npm install partytracks @cloudflare/calls
|
||||
```
|
||||
|
||||
**Raw API:** Direct HTTP + WebRTC for custom requirements (documented below).
|
||||
|
||||
## Endpoints
|
||||
|
||||
### Create Session
|
||||
```http
|
||||
POST /v1/apps/{appId}/sessions/new
|
||||
→ {sessionId, sessionDescription}
|
||||
```
|
||||
|
||||
### Add Track (Publish)
|
||||
```http
|
||||
POST /v1/apps/{appId}/sessions/{sessionId}/tracks/new
|
||||
Body: {
|
||||
sessionDescription: {sdp, type: "offer"},
|
||||
tracks: [{location: "local", trackName: "my-video"}]
|
||||
}
|
||||
→ {sessionDescription, tracks: [{trackName}]}
|
||||
```
|
||||
|
||||
### Add Track (Subscribe)
|
||||
```http
|
||||
POST /v1/apps/{appId}/sessions/{sessionId}/tracks/new
|
||||
Body: {
|
||||
tracks: [{
|
||||
location: "remote",
|
||||
trackName: "remote-track-id",
|
||||
sessionId: "other-session-id"
|
||||
}]
|
||||
}
|
||||
→ {sessionDescription} (server offer)
|
||||
```
|
||||
|
||||
### Renegotiate
|
||||
```http
|
||||
PUT /v1/apps/{appId}/sessions/{sessionId}/renegotiate
|
||||
Body: {sessionDescription: {sdp, type: "answer"}}
|
||||
```
|
||||
|
||||
### Close Tracks
|
||||
```http
|
||||
PUT /v1/apps/{appId}/sessions/{sessionId}/tracks/close
|
||||
Body: {tracks: [{trackName}]}
|
||||
→ {requiresImmediateRenegotiation: boolean}
|
||||
```
|
||||
|
||||
### Get Session
|
||||
```http
|
||||
GET /v1/apps/{appId}/sessions/{sessionId}
|
||||
→ {sessionId, tracks: TrackMetadata[]}
|
||||
```
|
||||
|
||||
## TypeScript Types
|
||||
|
||||
```typescript
|
||||
interface TrackMetadata {
|
||||
trackName: string;
|
||||
location: "local" | "remote";
|
||||
sessionId?: string; // For remote tracks
|
||||
mid?: string; // WebRTC mid
|
||||
}
|
||||
```
|
||||
|
||||
## WebRTC Flow
|
||||
|
||||
```typescript
|
||||
// 1. Create PeerConnection
|
||||
const pc = new RTCPeerConnection({
|
||||
iceServers: [{urls: 'stun:stun.cloudflare.com:3478'}]
|
||||
});
|
||||
|
||||
// 2. Add tracks
|
||||
const stream = await navigator.mediaDevices.getUserMedia({video: true, audio: true});
|
||||
stream.getTracks().forEach(track => pc.addTrack(track, stream));
|
||||
|
||||
// 3. Create offer
|
||||
const offer = await pc.createOffer();
|
||||
await pc.setLocalDescription(offer);
|
||||
|
||||
// 4. Send to backend → Cloudflare API
|
||||
const response = await fetch('/api/new-session', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({sdp: offer.sdp})
|
||||
});
|
||||
|
||||
// 5. Set remote answer
|
||||
const {sessionDescription} = await response.json();
|
||||
await pc.setRemoteDescription(sessionDescription);
|
||||
```
|
||||
|
||||
## Publishing
|
||||
|
||||
```typescript
|
||||
const offer = await pc.createOffer();
|
||||
await pc.setLocalDescription(offer);
|
||||
|
||||
const res = await fetch(`/api/sessions/${sessionId}/tracks`, {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({
|
||||
sdp: offer.sdp,
|
||||
tracks: [{location: 'local', trackName: 'my-video'}]
|
||||
})
|
||||
});
|
||||
|
||||
const {sessionDescription, tracks} = await res.json();
|
||||
await pc.setRemoteDescription(sessionDescription);
|
||||
const publishedTrackId = tracks[0].trackName; // Share with others
|
||||
```
|
||||
|
||||
## Subscribing
|
||||
|
||||
```typescript
|
||||
const res = await fetch(`/api/sessions/${sessionId}/tracks`, {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({
|
||||
tracks: [{location: 'remote', trackName: remoteTrackId, sessionId: remoteSessionId}]
|
||||
})
|
||||
});
|
||||
|
||||
const {sessionDescription} = await res.json();
|
||||
await pc.setRemoteDescription(sessionDescription);
|
||||
|
||||
const answer = await pc.createAnswer();
|
||||
await pc.setLocalDescription(answer);
|
||||
|
||||
await fetch(`/api/sessions/${sessionId}/renegotiate`, {
|
||||
method: 'PUT',
|
||||
body: JSON.stringify({sdp: answer.sdp})
|
||||
});
|
||||
|
||||
pc.ontrack = (event) => {
|
||||
const [remoteStream] = event.streams;
|
||||
videoElement.srcObject = remoteStream;
|
||||
};
|
||||
```
|
||||
@@ -0,0 +1,137 @@
|
||||
# Configuration & Deployment
|
||||
|
||||
## Dashboard Setup
|
||||
|
||||
1. Navigate to https://dash.cloudflare.com/?to=/:account/calls
|
||||
2. Click "Create Application" (or use existing app)
|
||||
3. Copy `CALLS_APP_ID` from dashboard
|
||||
4. Generate and copy `CALLS_APP_SECRET` (treat as sensitive credential)
|
||||
5. Use credentials in Wrangler config or environment variables below
|
||||
|
||||
## Dependencies
|
||||
|
||||
**Backend (Workers):** Built-in fetch API, no additional packages required
|
||||
|
||||
**Client (PartyTracks):**
|
||||
```bash
|
||||
npm install partytracks @cloudflare/calls
|
||||
```
|
||||
|
||||
**Client (React + PartyTracks):**
|
||||
```bash
|
||||
npm install partytracks @cloudflare/calls observable-hooks
|
||||
# Observable hooks: useObservableAsValue, useValueAsObservable
|
||||
```
|
||||
|
||||
**Client (Raw API):** Native browser WebRTC API only
|
||||
|
||||
## Wrangler Setup
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"name": "my-calls-app",
|
||||
"main": "src/index.ts",
|
||||
"compatibility_date": "2025-01-01", // Use current date for new projects
|
||||
"vars": {
|
||||
"CALLS_APP_ID": "your-app-id",
|
||||
"MAX_WEBCAM_BITRATE": "1200000",
|
||||
"MAX_WEBCAM_FRAMERATE": "24",
|
||||
"MAX_WEBCAM_QUALITY_LEVEL": "1080"
|
||||
},
|
||||
// Set secret: wrangler secret put CALLS_APP_SECRET
|
||||
"durable_objects": {
|
||||
"bindings": [
|
||||
{
|
||||
"name": "ROOM",
|
||||
"class_name": "Room"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
wrangler login
|
||||
wrangler secret put CALLS_APP_SECRET
|
||||
wrangler deploy
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
**Required:**
|
||||
- `CALLS_APP_ID`: From dashboard
|
||||
- `CALLS_APP_SECRET`: From dashboard (secret)
|
||||
|
||||
**Optional:**
|
||||
- `MAX_WEBCAM_BITRATE` (default: 1200000)
|
||||
- `MAX_WEBCAM_FRAMERATE` (default: 24)
|
||||
- `MAX_WEBCAM_QUALITY_LEVEL` (default: 1080)
|
||||
- `TURN_SERVICE_ID`: TURN service
|
||||
- `TURN_SERVICE_TOKEN`: TURN auth (secret)
|
||||
|
||||
## TURN Configuration
|
||||
|
||||
```javascript
|
||||
const pc = new RTCPeerConnection({
|
||||
iceServers: [
|
||||
{ urls: 'stun:stun.cloudflare.com:3478' },
|
||||
{
|
||||
urls: [
|
||||
'turn:turn.cloudflare.com:3478?transport=udp',
|
||||
'turn:turn.cloudflare.com:3478?transport=tcp',
|
||||
'turns:turn.cloudflare.com:5349?transport=tcp'
|
||||
],
|
||||
username: turnUsername,
|
||||
credential: turnCredential
|
||||
}
|
||||
],
|
||||
bundlePolicy: 'max-bundle', // Recommended: reduces overhead
|
||||
iceTransportPolicy: 'all' // Use 'relay' to force TURN (testing only)
|
||||
});
|
||||
```
|
||||
|
||||
**Ports:** 3478 (UDP/TCP), 53 (UDP), 80 (TCP), 443 (TLS), 5349 (TLS)
|
||||
|
||||
**When to use TURN:** Required for restrictive corporate firewalls/networks that block UDP. ~5-10% of connections fallback to TURN. STUN works for most users.
|
||||
|
||||
**ICE candidate filtering:** Cloudflare handles candidate filtering automatically. No need to manually filter candidates.
|
||||
|
||||
## Durable Object Boilerplate
|
||||
|
||||
Minimal presence system:
|
||||
|
||||
```typescript
|
||||
export class Room {
|
||||
private sessions = new Map<string, {userId: string, tracks: string[]}>();
|
||||
|
||||
async fetch(req: Request) {
|
||||
const {pathname} = new URL(req.url);
|
||||
const body = await req.json();
|
||||
|
||||
if (pathname === '/join') {
|
||||
this.sessions.set(body.sessionId, {userId: body.userId, tracks: []});
|
||||
return Response.json({participants: this.sessions.size});
|
||||
}
|
||||
|
||||
if (pathname === '/publish') {
|
||||
this.sessions.get(body.sessionId)?.tracks.push(...body.tracks);
|
||||
// Broadcast to others via WebSocket (not shown)
|
||||
return new Response('OK');
|
||||
}
|
||||
|
||||
return new Response('Not found', {status: 404});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Environment Validation
|
||||
|
||||
Check credentials before first API call:
|
||||
|
||||
```typescript
|
||||
if (!env.CALLS_APP_ID || !env.CALLS_APP_SECRET) {
|
||||
throw new Error('CALLS_APP_ID and CALLS_APP_SECRET required');
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,133 @@
|
||||
# Gotchas & Troubleshooting
|
||||
|
||||
## Common Errors
|
||||
|
||||
### "Slow initial connect (~1.8s)"
|
||||
|
||||
**Cause:** First STUN delayed during consensus forming (normal behavior)
|
||||
**Solution:** Subsequent connections are faster. CF detects DTLS ClientHello early to compensate.
|
||||
|
||||
### "No media flow"
|
||||
|
||||
**Cause:** SDP exchange incomplete, connection not established, tracks not added before offer, browser permissions missing
|
||||
**Solution:**
|
||||
1. Verify SDP exchange complete
|
||||
2. Check `pc.connectionState === 'connected'`
|
||||
3. Ensure tracks added before creating offer
|
||||
4. Confirm browser permissions granted
|
||||
5. Use `chrome://webrtc-internals` for debugging
|
||||
|
||||
### "Track not receiving"
|
||||
|
||||
**Cause:** Track not published, track ID not shared, session IDs mismatch, `pc.ontrack` not set, renegotiation needed
|
||||
**Solution:**
|
||||
1. Verify track published successfully
|
||||
2. Confirm track ID shared between peers
|
||||
3. Check session IDs match
|
||||
4. Set `pc.ontrack` handler before answer
|
||||
5. Trigger renegotiation if needed
|
||||
|
||||
### "ICE connection failed"
|
||||
|
||||
**Cause:** Network changed, firewall blocked UDP, TURN needed, transient network issue
|
||||
**Solution:**
|
||||
```typescript
|
||||
pc.oniceconnectionstatechange = async () => {
|
||||
if (pc.iceConnectionState === 'failed') {
|
||||
console.warn('ICE failed, attempting restart');
|
||||
await pc.restartIce(); // Triggers new ICE gathering
|
||||
|
||||
// Create new offer with ICE restart flag
|
||||
const offer = await pc.createOffer({iceRestart: true});
|
||||
await pc.setLocalDescription(offer);
|
||||
|
||||
// Send to backend → Cloudflare API
|
||||
await fetch(`/api/sessions/${sessionId}/renegotiate`, {
|
||||
method: 'PUT',
|
||||
body: JSON.stringify({sdp: offer.sdp})
|
||||
});
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### "Track stuck/frozen"
|
||||
|
||||
**Cause:** Sender paused track, network congestion, codec mismatch, mobile browser backgrounded
|
||||
**Solution:**
|
||||
1. Check `track.enabled` and `track.readyState === 'live'`
|
||||
2. Verify sender active: `pc.getSenders().find(s => s.track === track)`
|
||||
3. Check stats for packet loss/jitter (see patterns.md)
|
||||
4. On mobile: Re-acquire tracks when app foregrounded
|
||||
5. Test with different codecs if persistent
|
||||
|
||||
### "Network change disconnects call"
|
||||
|
||||
**Cause:** Mobile switching WiFi↔cellular, laptop changing networks
|
||||
**Solution:**
|
||||
```typescript
|
||||
// Listen for network changes
|
||||
if ('connection' in navigator) {
|
||||
(navigator as any).connection.addEventListener('change', async () => {
|
||||
console.log('Network changed');
|
||||
await pc.restartIce(); // Use ICE restart pattern above
|
||||
});
|
||||
}
|
||||
|
||||
// Or use PartyTracks (handles automatically)
|
||||
```
|
||||
|
||||
## Retry with Exponential Backoff
|
||||
|
||||
```typescript
|
||||
async function fetchWithRetry(url: string, options: RequestInit, maxRetries = 3) {
|
||||
for (let i = 0; i < maxRetries; i++) {
|
||||
try {
|
||||
const res = await fetch(url, options);
|
||||
if (res.ok) return res;
|
||||
if (res.status >= 500) throw new Error('Server error');
|
||||
return res; // Client error, don't retry
|
||||
} catch (err) {
|
||||
if (i === maxRetries - 1) throw err;
|
||||
const delay = Math.min(1000 * 2 ** i, 10000); // Cap at 10s
|
||||
await new Promise(resolve => setTimeout(resolve, delay));
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Debugging with chrome://webrtc-internals
|
||||
|
||||
1. Open `chrome://webrtc-internals` in Chrome/Edge
|
||||
2. Find your PeerConnection in the list
|
||||
3. Check **Stats graphs** for packet loss, jitter, bandwidth
|
||||
4. Check **ICE candidate pairs**: Look for `succeeded` state, relay vs host candidates
|
||||
5. Check **getStats**: Raw metrics for inbound/outbound RTP
|
||||
6. Look for errors in **Event log**: `iceConnectionState`, `connectionState` changes
|
||||
7. Export data with "Download the PeerConnection updates and stats data" button
|
||||
8. Common issues visible here: ICE failures, high packet loss, bitrate drops
|
||||
|
||||
## Limits
|
||||
|
||||
| Resource/Limit | Value | Notes |
|
||||
|----------------|-------|-------|
|
||||
| Egress (Free) | 1TB/month | Per account |
|
||||
| Egress (Paid) | $0.05/GB | After free tier |
|
||||
| Inbound traffic | Free | All plans |
|
||||
| TURN service | Free | Included with SFU |
|
||||
| Participants | No hard limit | Client bandwidth/CPU bound (typically 10-50 tracks) |
|
||||
| Tracks per session | No hard limit | Client resources limited |
|
||||
| Session duration | No hard limit | Production calls run for hours |
|
||||
| WebRTC ports | UDP 1024-65535 | Outbound only, required for media |
|
||||
| API rate limit | 600 req/min | Per app, burst allowed |
|
||||
|
||||
## Security Checklist
|
||||
|
||||
- ✅ **Never expose** `CALLS_APP_SECRET` to client
|
||||
- ✅ **Validate user identity** in backend before creating sessions
|
||||
- ✅ **Implement auth tokens** for session access (JWT in custom header)
|
||||
- ✅ **Rate limit** session creation endpoints
|
||||
- ✅ **Expire sessions** server-side after inactivity
|
||||
- ✅ **Validate track IDs** before subscribing (prevent unauthorized access)
|
||||
- ✅ **Use HTTPS** for all signaling (API calls)
|
||||
- ✅ **Enable DTLS-SRTP** (automatic with Cloudflare, encrypts media)
|
||||
- ⚠️ **Consider E2EE** for sensitive content (implement client-side with Insertable Streams API)
|
||||
@@ -0,0 +1,174 @@
|
||||
# Patterns & Use Cases
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Client (WebRTC) <---> CF Edge <---> Backend (HTTP)
|
||||
|
|
||||
CF Backbone (310+ DCs)
|
||||
|
|
||||
Other Edges <---> Other Clients
|
||||
```
|
||||
|
||||
Anycast: Last-mile <50ms (95%), no region select, NACK shield, distributed consensus
|
||||
|
||||
Cascading trees auto-scale to millions:
|
||||
```
|
||||
Publisher -> Edge A -> Edge B -> Sub1
|
||||
\-> Edge C -> Sub2,3
|
||||
```
|
||||
|
||||
## Use Cases
|
||||
|
||||
**1:1:** A creates session+publishes, B creates+subscribes to A+publishes, A subscribes to B
|
||||
**N:N:** All create session+publish, backend broadcasts track IDs, all subscribe to others
|
||||
**1:N:** Publisher creates+publishes, viewers each create+subscribe (no fan-out limit)
|
||||
**Breakout:** Same PeerConnection! Backend closes/adds tracks, no recreation
|
||||
|
||||
## PartyTracks (Recommended)
|
||||
|
||||
Observable-based client with automatic device/network handling:
|
||||
|
||||
```typescript
|
||||
import {PartyTracks} from 'partytracks';
|
||||
|
||||
// Create client
|
||||
const pt = new PartyTracks({
|
||||
apiUrl: '/api/calls',
|
||||
sessionId: 'my-session',
|
||||
onTrack: (track, peer) => {
|
||||
const video = document.getElementById(`video-${peer.id}`) as HTMLVideoElement;
|
||||
video.srcObject = new MediaStream([track]);
|
||||
}
|
||||
});
|
||||
|
||||
// Publish camera (push API)
|
||||
const camera = await pt.getCamera(); // Auto-requests permissions, handles device changes
|
||||
await pt.publishTrack(camera, {trackName: 'my-camera'});
|
||||
|
||||
// Subscribe to remote track (pull API)
|
||||
await pt.subscribeToTrack({trackName: 'remote-camera', sessionId: 'other-session'});
|
||||
|
||||
// React hook example
|
||||
import {useObservableAsValue} from 'observable-hooks';
|
||||
|
||||
function VideoCall() {
|
||||
const localTracks = useObservableAsValue(pt.localTracks$);
|
||||
const remoteTracks = useObservableAsValue(pt.remoteTracks$);
|
||||
|
||||
return <div>{/* Render tracks */}</div>;
|
||||
}
|
||||
|
||||
// Screenshare
|
||||
const screen = await pt.getScreenshare();
|
||||
await pt.publishTrack(screen, {trackName: 'my-screen'});
|
||||
|
||||
// Handle device changes (automatic)
|
||||
// PartyTracks detects device changes (e.g., Bluetooth headset) and renegotiates
|
||||
```
|
||||
|
||||
## Backend
|
||||
|
||||
Express:
|
||||
```js
|
||||
app.post('/api/new-session', async (req, res) => {
|
||||
const r = await fetch(`${CALLS_API}/apps/${process.env.CALLS_APP_ID}/sessions/new`,
|
||||
{method: 'POST', headers: {'Authorization': `Bearer ${process.env.CALLS_APP_SECRET}`}});
|
||||
res.json(await r.json());
|
||||
});
|
||||
```
|
||||
|
||||
Workers: Same pattern, use `env.CALLS_APP_ID` and `env.CALLS_APP_SECRET`
|
||||
|
||||
DO Presence: See configuration.md for boilerplate
|
||||
|
||||
## Audio Level Detection
|
||||
|
||||
```typescript
|
||||
// Attach analyzer to audio track
|
||||
function attachAudioLevelDetector(track: MediaStreamTrack) {
|
||||
const ctx = new AudioContext();
|
||||
const analyzer = ctx.createAnalyser();
|
||||
const src = ctx.createMediaStreamSource(new MediaStream([track]));
|
||||
src.connect(analyzer);
|
||||
|
||||
const data = new Uint8Array(analyzer.frequencyBinCount);
|
||||
const checkLevel = () => {
|
||||
analyzer.getByteFrequencyData(data);
|
||||
const level = data.reduce((a, b) => a + b) / data.length;
|
||||
if (level > 30) console.log('Speaking:', level); // Trigger UI update
|
||||
requestAnimationFrame(checkLevel);
|
||||
};
|
||||
checkLevel();
|
||||
}
|
||||
```
|
||||
|
||||
## Connection Quality Monitoring
|
||||
|
||||
```typescript
|
||||
pc.getStats().then(stats => {
|
||||
stats.forEach(report => {
|
||||
if (report.type === 'inbound-rtp' && report.kind === 'video') {
|
||||
const {packetsLost, packetsReceived, jitter} = report;
|
||||
const lossRate = packetsLost / (packetsLost + packetsReceived);
|
||||
if (lossRate > 0.05) console.warn('High packet loss:', lossRate);
|
||||
if (jitter > 100) console.warn('High jitter:', jitter);
|
||||
}
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## Stage Management (Limit Visible Participants)
|
||||
|
||||
```typescript
|
||||
// Subscribe to top 6 active speakers only
|
||||
let activeSubscriptions = new Set<string>();
|
||||
|
||||
function updateStage(topSpeakers: string[]) {
|
||||
const toAdd = topSpeakers.filter(id => !activeSubscriptions.has(id)).slice(0, 6);
|
||||
const toRemove = [...activeSubscriptions].filter(id => !topSpeakers.includes(id));
|
||||
|
||||
toRemove.forEach(id => {
|
||||
pc.getSenders().find(s => s.track?.id === id)?.track?.stop();
|
||||
activeSubscriptions.delete(id);
|
||||
});
|
||||
|
||||
toAdd.forEach(async id => {
|
||||
await fetch(`/api/subscribe`, {method: 'POST', body: JSON.stringify({trackId: id})});
|
||||
activeSubscriptions.add(id);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## Advanced
|
||||
|
||||
Bandwidth mgmt:
|
||||
```ts
|
||||
const s = pc.getSenders().find(s => s.track?.kind === 'video');
|
||||
const p = s.getParameters();
|
||||
if (!p.encodings) p.encodings = [{}];
|
||||
p.encodings[0].maxBitrate = 1200000; p.encodings[0].maxFramerate = 24;
|
||||
await s.setParameters(p);
|
||||
```
|
||||
|
||||
Simulcast (CF auto-forwards best layer):
|
||||
```ts
|
||||
pc.addTransceiver('video', {direction: 'sendonly', sendEncodings: [
|
||||
{rid: 'high', maxBitrate: 1200000},
|
||||
{rid: 'med', maxBitrate: 600000, scaleResolutionDownBy: 2},
|
||||
{rid: 'low', maxBitrate: 200000, scaleResolutionDownBy: 4}
|
||||
]});
|
||||
```
|
||||
|
||||
DataChannel:
|
||||
```ts
|
||||
const dc = pc.createDataChannel('chat', {ordered: true, maxRetransmits: 3});
|
||||
dc.onopen = () => dc.send(JSON.stringify({type: 'chat', text: 'Hi'}));
|
||||
dc.onmessage = (e) => console.log('RX:', JSON.parse(e.data));
|
||||
```
|
||||
|
||||
**WHIP/WHEP:** For streaming interop (OBS → SFU, SFU → video players), use WHIP (ingest) and WHEP (egress) protocols. See Cloudflare Stream integration docs.
|
||||
|
||||
Integrations: R2 for recording `env.R2_BUCKET.put(...)`, Queues for analytics
|
||||
|
||||
Perf: 100-250ms connect, ~50ms latency (95%), 200-400ms glass-to-glass, no participant limit (client: 10-50 tracks)
|
||||
Reference in New Issue
Block a user