The Signal

1. The problem

   Keeping up with a handful of topics means reading twenty tabs. The useful version is a short digest built from what was published today — and an LLM product like that is cheap to demo and expensive to run unless quotas, caching and failure handling are designed in from day one.

2. The approach

   • Firecrawl pulls live pages for the chosen topics; OpenRouter turns them into a digest; Zod validates the model's JSON before anything is stored.
   • Cost controls as product rules: Clerk-authenticated users only, three digests per user per day, per-account and per-IP rate limits, and a three-hour cache so a repeated topic reuses the crawl.
   • Resilience: model fallback rotation when an endpoint is down, one retry on upstream failure, a health endpoint that checks keys and model availability.
   • Digests persist in Convex with shareable links and a de-duplicated "crawled today" history; scheduled jobs purge expired ones.

3. What shipped

   • `POST /api/digest` with topic allow-listing, quota and rate-limit checks.
   • Convex schema and functions for digests, rate limits, subscribers and an email queue.
   • Shareable snapshot pages; cron-driven cleanup.

Faheem Musthafa · 2026