AI Rules for Video Streaming

AI Serves Video Like a File Download

AI generates video delivery with: a single file served as a download (the entire 500MB video must download before playback starts), no adaptive bitrate (4K video over a 3G connection — infinite buffering), no CDN (video served from the application server — bandwidth saturated at 5 concurrent viewers), no transcoding (one quality level, take it or leave it), and no seeking support (user clicks to the middle of a video — the server sends the entire file from the beginning). Video looks simple until you serve it to real users on real networks.

Modern video streaming is: adaptive (HLS or DASH automatically switches quality based on network speed), CDN-delivered (video segments served from edge locations worldwide), multi-quality (transcoded to 360p, 720p, 1080p, and 4K — the player selects the best quality for the connection), seekable (HTTP range requests let the player download only the requested segment), and thumbnail-previewed (hover over the timeline to see preview thumbnails). AI generates none of these.

These rules cover: HLS adaptive bitrate streaming, CDN video delivery, FFmpeg transcoding pipelines, thumbnail sprite generation, HTTP range request handling, and video player integration.

Rule 1: HLS Adaptive Bitrate Streaming

The rule: 'Use HTTP Live Streaming (HLS) for video delivery. HLS splits the video into small segments (2-10 seconds each) and provides a playlist (m3u8 file) listing all segments. The player downloads segments sequentially, switching between quality levels based on network conditions. Slow connection: player downloads 360p segments. Fast connection: player downloads 1080p segments. Network degrades mid-video: player drops to lower quality seamlessly — no rebuffering.'

For the HLS structure: 'A master playlist (master.m3u8) lists available quality levels with bandwidth and resolution. Each quality level has its own playlist (720p.m3u8) listing its segments. The player: (1) downloads master.m3u8, (2) selects the highest quality the connection can sustain, (3) downloads segments from that quality playlist, (4) monitors download speed and switches qualities as needed. All over standard HTTP — no special streaming server needed, any CDN can serve HLS.'

AI generates: <video src="/videos/lecture.mp4"> — a single 500MB file. On a 5Mbps connection: 13 minutes to download before playback starts. On a 50Mbps connection: 4K video plays but wastes bandwidth for users who do not need 4K. HLS: playback starts in 2 seconds (first segment only), quality adapts to the connection, and seeking loads only the requested segment. Same video, fundamentally different delivery.

• HLS: video split into 2-10 second segments, served over standard HTTP
• Master playlist (m3u8) lists quality levels: 360p, 720p, 1080p, 4K
• Adaptive: player switches quality based on real-time network speed
• Playback starts in seconds: first segment only, not entire file
• Standard HTTP — any CDN can serve HLS, no special streaming server

Rule 2: FFmpeg Transcoding Pipeline

The rule: 'Transcode uploaded videos into multiple quality levels using FFmpeg. Quality ladder: 360p (640x360, 800kbps), 720p (1280x720, 2.5Mbps), 1080p (1920x1080, 5Mbps), 4K (3840x2160, 15Mbps — optional). Codec: H.264 for maximum compatibility, H.265/HEVC for better compression (50% smaller at same quality), AV1 for the future (even smaller, growing browser support). Generate HLS segments: ffmpeg -i input.mp4 -hls_time 6 -hls_playlist_type vod -hls_segment_filename "segment_%03d.ts" playlist.m3u8.'

For the async pipeline: 'Transcoding is CPU-intensive: a 1-hour video takes 30-60 minutes to transcode. Process in a background job: (1) upload original to S3, (2) create a transcode job, (3) dedicated transcoding worker (high-CPU instance) downloads, transcodes to all quality levels, generates HLS playlists, (4) uploads segments and playlists to S3/CDN, (5) updates the video record with the HLS URL. Cloud alternatives: AWS MediaConvert, Cloudflare Stream, or Mux — managed transcoding without running FFmpeg yourself.'

AI generates: serves the original uploaded video file directly. A 4K screencast at 100Mbps bitrate: unwatchable on mobile connections. A phone recording at 720p: looks fine but wastes bandwidth serving the original encoding (often inefficient). Transcoding normalizes everything: consistent quality levels, optimized bitrates, and HLS segments. The viewer always gets the right quality for their connection.

Rule 3: CDN Edge Delivery for Video

The rule: 'Serve video segments from a CDN, not from your application server. Video bandwidth: a single 1080p viewer consumes 5Mbps. 100 concurrent viewers: 500Mbps — more than most application servers can handle. A CDN: serves from 300+ edge locations (viewers download from the nearest edge), caches segments aggressively (HLS segments are immutable — Cache-Control: public, max-age=31536000), and handles bandwidth scaling automatically (1 viewer or 10,000 viewers, same cost model).'

For signed URLs: 'For paid or private content, use signed URLs that expire: generate a signed CDN URL with a 4-hour TTL when the player loads. The viewer can watch for 4 hours; after that, the URL expires. Sharing the URL does not work after expiry. AWS CloudFront signed cookies: sign the entire HLS path, all segments are accessible through the signed session. This is DRM-lite — prevents casual sharing without the complexity of full DRM (Widevine, FairPlay).'

AI generates: video served from the application server over a single origin connection. 10 concurrent viewers: the server is saturated. 100 viewers: the server is unreachable. With CDN: 10,000 concurrent viewers served from edge locations, the origin server handles zero video traffic (segments are cached). Video is the highest-bandwidth content type — CDN delivery is not optional, it is essential.

Rule 4: Thumbnail Sprite Generation

The rule: 'Generate thumbnail sprites for timeline preview: extract a frame every 5-10 seconds, combine into a sprite sheet (grid of thumbnails in one image), and provide a VTT (WebVTT) file mapping timestamps to sprite coordinates. The player: on timeline hover, reads the VTT, loads the sprite sheet, and shows the thumbnail at the hovered timestamp. This lets viewers visually scan through the video without playing it.'

For FFmpeg thumbnail extraction: 'Extract frames: ffmpeg -i input.mp4 -vf "fps=1/5,scale=160:90" -q:v 5 thumb_%04d.jpg — one frame every 5 seconds at 160x90. Combine into sprite: use sharp or ImageMagick to tile thumbnails into a grid (10x10 = 100 thumbnails per sprite). Generate VTT: map each thumbnail to its timestamp and sprite coordinates. Libraries: @mux/blurhash for placeholder generation, or built-in with Cloudflare Stream and Mux.'

AI generates: no thumbnails. The video timeline is a blank progress bar. The user clicks randomly, waits for the segment to load, realizes it is the wrong part, clicks again. With thumbnail preview: the user hovers over the timeline, sees exactly where they are going, and clicks once to the right spot. One feature that dramatically improves video navigation UX.

• Extract frame every 5-10 seconds: FFmpeg fps filter at 160x90 resolution
• Sprite sheet: grid of thumbnails in one image — one HTTP request for all previews
• WebVTT file: maps timestamps to sprite coordinates for the player
• Timeline hover: player shows the thumbnail at the hovered position
• Managed services: Mux, Cloudflare Stream generate sprites automatically

Rule 5: Range Requests and Player Integration

The rule: 'Support HTTP range requests for video seeking. The player sends: Range: bytes=1000000-2000000. The server responds: 206 Partial Content with only the requested byte range. This enables: seeking (jump to any point without downloading everything before it), resume (interrupted downloads continue from where they stopped), and bandwidth efficiency (only download what is being watched). S3 and CDNs handle range requests natively — no application code needed.'

For player integration: 'Use hls.js for HLS playback in browsers that do not support HLS natively (all browsers except Safari). Pattern: if (Hls.isSupported()) { const hls = new Hls(); hls.loadSource("https://cdn.example.com/video/master.m3u8"); hls.attachMedia(videoElement); } else if (videoElement.canPlayType("application/vnd.apple.mpegurl")) { videoElement.src = hlsUrl; }. Safari plays HLS natively; other browsers need hls.js. One conditional, universal HLS playback.'

AI generates: <video src="video.mp4"> with no range request handling from the application server (responds with 200 and the entire file for every request). The user seeks to minute 30: the server sends 500MB from the beginning. With range requests: the server sends only the requested 1MB segment. 500x less data transferred for a single seek operation. CDNs handle this automatically — just serve the video from S3/CDN, not from your application server.

Complete Video Streaming Rules Template

Consolidated rules for video streaming.

• HLS adaptive streaming: segments + playlists, quality adapts to network speed
• FFmpeg transcoding: 360p, 720p, 1080p quality ladder with H.264/H.265 codec
• Async transcoding: background worker or managed service (MediaConvert, Mux, Cloudflare Stream)
• CDN delivery: edge-cached segments, handles 10,000 concurrent viewers, origin serves zero
• Signed URLs for paid content: time-limited access, prevents casual URL sharing
• Thumbnail sprites: frame every 5s, sprite sheet + VTT, timeline hover preview
• Range requests: seek without downloading everything, 206 Partial Content
• hls.js for browser playback: Safari native HLS, all others via hls.js library