AI Rules for Image Processing

AI Processes Images Like It Has Infinite Memory

AI generates image processing with: synchronous processing in the request handler (user waits 10 seconds while a 20MB photo is resized), no size validation (accept any file size, crash on a 500MB upload), no format conversion (store the original JPEG/PNG, serve unoptimized), no thumbnail generation (resize on every request or serve the full image everywhere), no metadata stripping (EXIF data with GPS coordinates, camera model, and timestamps leaks to all viewers), and ImageMagick shell commands (command injection risk, no streaming, process-level crashes).

Modern image processing is: library-based (sharp — the fastest Node.js image processor, libvips under the hood), async-queued (background job processes the image, results stored in S3), format-optimized (convert to WebP/AVIF for 50% size reduction), multi-thumbnail (generate 3-5 sizes on upload, serve the right size per context), metadata-stripped (remove EXIF GPS, camera, and personal data), and upload-validated (max file size, allowed MIME types, magic byte verification). AI generates none of these.

These rules cover: sharp-based image pipelines, async processing queues, format conversion to WebP/AVIF, multi-size thumbnail generation, EXIF metadata stripping, and upload validation.

Rule 1: Sharp-Based Image Processing Pipelines

The rule: 'Use sharp for all server-side image processing. Sharp is: 4-5x faster than ImageMagick, streaming-capable (constant memory), and does not spawn shell processes (no command injection risk). Pipeline: await sharp(inputBuffer).resize(800, 600, { fit: "cover" }).webp({ quality: 80 }).toBuffer(). Chained operations: resize, crop, rotate, format convert, and compress in a single pipeline. Sharp processes the image in a single pass through libvips — no intermediate files.'

For pipeline composition: 'Build reusable pipeline functions: async function processAvatar(input: Buffer): Promise<Buffer> { return sharp(input).resize(200, 200, { fit: "cover", position: "attention" }).webp({ quality: 85 }).toBuffer(); }. The "attention" position crops to the most visually interesting area (faces, high-contrast regions). Each image type (avatar, hero, thumbnail, og-image) has its own pipeline function with appropriate dimensions, quality, and format.'

AI generates: const result = await exec('convert input.jpg -resize 800x600 output.jpg') — spawns a shell process (command injection if filename is user-controlled), creates temporary files on disk (slow I/O), and crashes the entire Node.js process if ImageMagick segfaults. Sharp: in-process, streaming, no shell, no temp files, no segfault risk. Same output, 5x faster, zero security risk.

• sharp: 4-5x faster than ImageMagick, streaming, no shell processes
• Single-pass pipeline: resize + crop + convert + compress in one operation
• fit options: cover (fill area, crop excess), contain (fit within, add padding), inside (fit within, no padding)
• position: attention (smart crop to interesting area), centre, entropy
• Reusable pipeline functions: processAvatar(), processHero(), processThumbnail()

Rule 2: Async Background Processing

The rule: 'Process images in a background job, not in the upload request handler. Flow: (1) upload handler validates the file and stores the original in S3, (2) publishes an image.uploaded event with the S3 key, (3) returns 202 Accepted with the image ID immediately, (4) background worker downloads from S3, processes through the sharp pipeline, uploads processed versions back to S3, (5) updates the database record with processed image URLs. The user sees a placeholder while processing completes (1-3 seconds).'

For the placeholder pattern: 'While the image processes, serve a blurred low-quality placeholder (LQIP). Generate it synchronously during upload (20x20 pixels, base64-encoded, 200 bytes): const lqip = await sharp(buffer).resize(20, 20).blur(10).toBuffer(); Store as a base64 data URI on the record. The frontend shows the LQIP with a blur-to-sharp transition when the full image is ready. Users see immediate visual feedback instead of a loading spinner.'

AI generates: const processed = await sharp(buffer).resize(800).toBuffer(); in the upload handler. A 20MB photo takes 3 seconds to process. The user waits. 10 concurrent uploads: 10 sharp operations competing for CPU, all slow down. Background processing: upload returns in 200ms, processing happens in a dedicated worker with controlled concurrency (max 4 simultaneous operations). User experience: instant upload, images appear in 1-3 seconds.

Rule 3: WebP/AVIF Output Formats

The rule: 'Convert uploaded images to WebP (primary) and optionally AVIF (maximum compression). Store both: the original (for future reprocessing) and the optimized version (for serving). WebP quality 80: visually identical to JPEG quality 90 at 35% smaller file size. AVIF quality 65: visually identical at 50% smaller. Generate both formats in the processing pipeline: const webp = await sharp(buffer).webp({ quality: 80 }).toBuffer(); const avif = await sharp(buffer).avif({ quality: 65 }).toBuffer().'

For serving strategy: 'Use content negotiation or the <picture> element to serve the optimal format. CDN-level: Cloudflare, Vercel, and imgix auto-negotiate format based on the Accept header. Application-level: check Accept header for image/avif, fall back to image/webp, then image/jpeg. Store all formats in S3: original.jpg, optimized.webp, optimized.avif. The CDN or application serves the best format the browser supports.'

AI generates: stores and serves the original JPEG/PNG. A 5MB JPEG profile photo served to every visitor. With WebP conversion: 1.5MB (same visual quality). With AVIF: 1MB. Over 1000 page views: 3.5GB saved per image. Across all images on the site: terabytes of bandwidth saved per month. Format conversion is the single highest-impact image optimization — and it runs once per image, not per request.

Rule 4: Multi-Size Thumbnail Generation

The rule: 'Generate multiple sizes on upload, not on request. Standard sizes: thumbnail (150x150, avatar/list), small (400x300, card), medium (800x600, article body), large (1200x900, hero/full-width), and original (preserved for future reprocessing). Generate all sizes in the background worker: const sizes = [{ name: "thumb", width: 150, height: 150 }, { name: "sm", width: 400, height: 300 }, ...]; const results = await Promise.all(sizes.map(s => sharp(buffer).resize(s.width, s.height, { fit: "cover" }).webp({ quality: 80 }).toBuffer())).'

For responsive serving: 'Store URLs for all sizes: { thumb: "s3://thumb.webp", sm: "s3://sm.webp", md: "s3://md.webp", lg: "s3://lg.webp" }. The frontend selects the appropriate size: avatar component uses thumb, article card uses sm, full article uses md, hero section uses lg. Using srcset: <img srcset="thumb.webp 150w, sm.webp 400w, md.webp 800w, lg.webp 1200w" sizes="(max-width: 640px) 100vw, 800px" />. The browser downloads only the size it needs.'

AI generates: one size stored, served everywhere. The 1200px hero image displayed at 150px in a thumbnail: 10x more data downloaded than needed. Or: resize on every request (sharp in the request handler — CPU-intensive, blocks the event loop). Pre-generated thumbnails: zero request-time processing, optimal size per context, and srcset delivers the right image to every device.

• Generate on upload, serve on request — zero request-time processing
• Standard sizes: thumb (150), sm (400), md (800), lg (1200), original (preserved)
• Promise.all for parallel generation — all sizes in one pipeline pass
• srcset with all sizes: browser picks optimal width automatically
• Store URLs per size on the record: { thumb, sm, md, lg, original }

Rule 5: EXIF Metadata Stripping and Upload Validation

The rule: 'Strip EXIF metadata from all user-uploaded images. EXIF data may contain: GPS coordinates (user home location), camera serial number (device fingerprinting), creation timestamp (when the photo was taken), and software information. Sharp strips metadata by default when converting formats. Explicitly: sharp(buffer).rotate().withMetadata(false).toBuffer(). The .rotate() applies EXIF orientation before stripping (prevents rotated images after metadata removal).'

For upload validation: 'Validate before processing: (1) file size limit (10MB for most applications, configurable per context), (2) MIME type check from Content-Type header (first pass — can be spoofed), (3) magic byte verification (read first bytes to confirm actual file type — a .jpg that starts with PK is actually a ZIP), (4) dimension limits (no 50,000x50,000 pixel images — they exhaust memory during processing). Reject invalid files before sharp touches them.'

AI generates: accepts any file (no size limit, no type check, no magic byte verification). A user uploads a 2GB file labeled as JPEG — sharp tries to process it, memory exhausted, process crashes. Or: a PHP file renamed to .jpg is stored and potentially served. Validation first: reject oversized files (413), reject wrong types (415), verify magic bytes (actual content matches claimed type). Only valid images reach the processing pipeline.

Complete Image Processing Rules Template

Consolidated rules for image processing.

• sharp for all processing: 4-5x faster than ImageMagick, streaming, no shell commands
• Async background processing: upload returns 202, worker processes, LQIP placeholder
• WebP (quality 80) + AVIF (quality 65): 35-50% smaller than JPEG at same visual quality
• Multi-size thumbnails on upload: thumb, sm, md, lg — srcset serves optimal size
• Strip EXIF metadata: GPS, camera serial, timestamps — .rotate() then .withMetadata(false)
• Upload validation: size limit, MIME check, magic byte verification, dimension limits
• Store original for reprocessing: if requirements change, regenerate from source
• LQIP placeholder: 20x20 base64, blur-to-sharp transition while full image loads