Case study
RAG Report Platform
A production-grade engine for retrieval-grounded, multi-source AI report generation - provider-abstracted, layer-cached, and honest when a dependency fails.
- TypeScript (strict)
- Next.js 15 (App Router)
- React 19
- PostgreSQL + pgvector
- Zod
- Vitest
- GitHub Actions + CodeQL
What it is
A working, tested system that generates structured, grounded AI reports from two ingredients: a curated knowledge corpus (retrieved with RAG) and live external data signals. It caches in layers so a warm request is near-instant and near-free, and it degrades honestly when any single dependency is down. Swap the corpus and the signal providers and the same engine writes agronomy advisories, environmental briefings, or logistics guidance.
Why it exists
Wiring a prompt to an API key is a demo. Production AI has to answer harder questions. How do you keep a model outage from taking down the product? How do you know a prompt change made things better, not worse? How do you keep per-request cost from drifting silently? How do you stop the model from being the slow, expensive part of every request? This repo is a set of answered questions.
Three ideas do most of the work
One function for every model call
Each call renders a versioned prompt template, goes through an injectable transport, validates output against a Zod schema with one corrective retry, and logs token counts and estimated cost - success or failure. Because the transport is injected, the whole test suite runs offline and never touches the network.
Cached by what determines the report
A durable base - expensive, rarely changing - is generated once under a double-generation lock and reused. Live signals layer on top. The warm path never calls the model, so a repeat request is near-instant and near-free.
The model is never a single point of failure
Retrieval returns empty on any error and generation still proceeds. Each signal provider degrades to null independently. A failed report is marked failed with an honest error and full provenance - never left hanging.
Request lifecycle
One path, with every dependency optional. Full detail in ARCHITECTURE.md and the twelve ADRs.
- 1
Report request
Zod-validated at the API route
- 2
Report pipeline
orchestrates cache, retrieval, signals
- 3
L1 base cache
hit → assemble · miss → generate under a lock
- 4
RAG retrieval → structured generation
graceful-empty; one AI client; usage + cost logged
- 5
Signal providers
each optional, degrade to null independently
- 6
Grounded report + provenance
or an honest, marked failure
What's inside
AI core
One provider-abstracted client, structured output with a corrective retry, per-call cost logging, and prompt configuration stored as versioned, audited data.
Report engine
An L1 durable base cache with a double-generation lock, an L2 live overlay with a deterministic fallback, and an orchestrator with honest failure handling and full provenance.
RAG
A pgvector corpus with an HNSW index and a match_chunks function, graceful-empty retrieval, a coverage-gap demand signal, and an offline-capable seeding CLI.
Signals
A SignalProvider seam with a deterministic temporal provider and a network-backed weather provider - each optional and independently degradable.
Data & security
Migrations as the single source of schema truth, RLS enabled in the same migration as each table and enforced by a CI gate, plus a secrets policy with gitleaks and an .env.example guard test.
Billing
Idempotent webhook processing via an event-id ledger, with entitlement mapping that fails closed.
Evaluation
An LLM-as-judge harness that scores a committed golden set against a fixed rubric, so a prompt change can be measured instead of guessed at.
Tooling
CI running typecheck, lint, test, RLS coverage, migration order, build, secret scan, and CodeQL - 54 offline tests on every push.
By the numbers
The AI transport targets any OpenAI-compatible endpoint over fetch - no vendor SDK coupling. The default runtime uses an in-memory store and cache, so the app runs with zero infrastructure; swapping in the Postgres implementations is a change to one composition file.
Explore the code
The repository is fully documented - an architecture overview, twelve ADRs, an API contract, and a CI/CD walkthrough.