Resource / Checklist

RAG Readiness Checklist

A pre-build checklist for retrieval-augmented generation in a SaaS product: corpus, chunking, evals, citations, and cost.

/ Outcome

What you walk away with.

A RAG plan you can execute without producing a fragile demo, with measurable accuracy from day one.

/ System

/01

Corpus and chunking

Decide what is in scope, who can access it, and how it is chunked before touching a model.

  • Source list
  • Per-tenant scoping
  • Chunk size
  • Refresh cadence

/02

Evaluation and cost

Build a small eval set from real questions and pin a per-user cost ceiling before launch.

  • Eval set
  • Citation correctness
  • Latency budget
  • Cost ceiling
/ Checklist

Run through the list.

  1. 01List every document type the model can read
  2. 02Define tenant boundaries and access checks
  3. 03Pick chunking strategy and embedding model
  4. 04Build a 50-question eval set from real users
  5. 05Add citations to every answer
  6. 06Set per-user and per-month cost caps
/ FAQ

Questions founders ask.

Do we need a vector database?

Often Postgres with pgvector is enough at the start. Move to a dedicated store when query patterns demand it.

How do we measure quality?

Run the eval set on every change and watch citation correctness, not just answer overlap.

Plan my RAG build